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Abstract 

A unified framework to obtain all known lower bounds (random coding, typical random coding and 
expurgated bound) on the reliability function of a point-to-point discrete memoryless channel (DMC) is 
presented. By using a similar idea for a two-user discrete memoryless (DM) multiple-access channel (MAC), 
three lower bounds on the reliability function are derived. The first one (random coding) is identical to 
the best known lower bound on the reliability function of DM-MAC. It is shown that the random coding 
bound is the performance of the average code in the constant composition code ensemble. The second 
bound (Typical random coding) is the typical performance of the constant composition code ensemble. To 
derive the third bound (expurgated), we eliminate some of the codewords from the codebook with larger 
rate. This is the first bound of this type that explicitly uses the method of expurgation for MACs. It is 
shown that the exponent of the typical random coding and the expurgated bounds are greater than or 
equal to the exponent of the known random coding bounds for all rate pairs. Moreover, an example is 
given where the exponent of the expurgated bound is strictly larger. All these bounds can be universally 
obtained for all discrete memoryless MACs with given input and output alphabets. 

I. Introduction 

In this paper, we consider the problem of communication over a multiple-access channel (MAC) without 
feedback in the discrete memoryless setting. In particular, we consider the error exponents for this channel 
model. In this model, two transmitters wish to communicate reliably two independent messages to a 
single decoder. A schematic is depicted in Figure [1] 
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Fig. 1. A schematic of two-user multiple-access channel 



Error exponents have been meticulously studied for point to point discrete memoryless channels 
(DMCs) in the literature HJ-Q- The optimum error exponent E(R) at some fixed transmission rate 
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R (also known as the channel reliability function) gives the decoding error probability exponential rate 
of decay as a function of block-length for the best sequence of codes. Lower and upper bounds on 
the channel reliability function for the DMC are known. A lower bound, known as the random coding 
exponent, was developed by Fano [3] by upper-bounding the average error probability over an ensemble 
of codes. This bound is loose at low rates. Gallager |8] demonstrated that the random coding bound is 
the true average error exponent for the random code ensemble. This result illustrates that the weakness 
of the random coding bound, at low rates, is not due to upper-bounding the ensemble average. Rather, 
this weakness is due to the fact that the best codes perform much better than the average, especially at 
low rates. The random coding exponent is further improved at low rates by the process of "expurgation" 
|9|-111J. The expurgated bound coincides with the upper bound on the reliability function at R = fl2l 
pg. 189]. Barg and Forney |13| investigated another lower bound for the binary symmetric channel (BSC), 
called the "typical" random coding bound. The authors showed that almost all codes in the standard 
random coding ensemble exhibit a performance that is as good as the one described by the typical random 
coding bound. In addition, they showed that the typical error exponent is larger than the random coding 
exponent and smaller than the expurgated exponent at low rates. Regarding discrete memoryless multiple- 
access channels (DM-MACs), stronger versions of Ahlswede and Liao's coding theorem [14J, |15], giving 
exponential upper and lower bounds for the error probability, were derived by several authors. Slepian 
and Wolf QU, Dyachkov Il7l , Gallager [18J, Pokorny and Wallmeier OH, and Liu and Hughes ]|20) studied 
upper bounds on the error probability. Haroutunian |21| and Nazari |22J-|24J studied lower bounds on 
the error probability. 

Comparing the state of the art in the study of error exponents for DMCs and DM-MACs, we observe 
that the latter is much less advanced. We believe the main difficulty in the study of error exponents for 
DM-MACs is the fact that error performance in a DM-MAC depends on the pair of codebooks (in the 
case of a two-user MAC) used by the two transmitters, while at the same time, each transmitter can only 
control its own codebook. This simple fact has important consequences. For instance, expurgation has 
not been studied in MAC, since by eliminating some of the "bad" codeword pairs, we may end up with 
a set of correlated input sequences, which is hard to analyze. In this paper, we develop two new lower 
bounds for the reliability function of DM-MACs. These bound outperform the bounds of [19], [20J. 

Toward this goal, we first revisit the point-to-point case and look at the techniques that are used for 
obtaining the lower bounds on the optimum error exponents. The techniques can be broadly classified 
into three categories. The first is the Gallager technique QQ. Although this yields expressions for the 
error exponents that are computationally easier to evaluate than others, the expressions themselves are 
harder to interpret. The second is the Csiszar-Korner technique [12). This technique gives more intuitive 
expressions for the error exponents in terms of optimization of an objective function involving information 
quantities over probability distributions. This approach is more amenable to generalization to multi-user 
channels. The third is the graph decomposition technique using a-decoding [25 j. a-decoding is a class 
of decoding procedures that includes maximum likelihood decoding and minimum entropy decoding. 
Although this technique gives a simpler derivation of the exponents, we believe that it is harder to 
generalize this to multi-user channels. All three classes of techniques give expressions for the random 
coding and expurgated exponents. The expressions obtained by the three techniques appear in different 
forms. 

In developing our main result, we first develop a new simpler technique for deriving the random coding 
and expurgated exponents for the point-to-point channel using a constant composition code ensemble 
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Fig. 2. Lower bounds on the reliability function for point-to-point channel (random coding —, typical random 
coding — , expurgated — ) 

with a-decoding. We present our results in the format given in |25|. This technique also gives upper 
bounds on the ensemble averages. As a bonus, we obtain the typical random coding exponent for this 
channel. This gives an exact characterization (lower and upper bounds that meet) of the error exponent 
of almost all codes in the ensemble. When specialized to the BSC, this reduces to the typical random 
coding bound of Barg and Forney |130- Fig. 2 shows the random coding, the typical random coding, 
and the expurgated bounds for a BSC with crossover probability p = 0.05, which is representative of the 
general case. All the three lower bounds are expressed as minimizations of a single objective function 
under different constraint sets. The reasons for looking at typical performance are two-fold. The first is 
that the average error exponent is in general smaller than the typical error exponent at low rates, hence 
the latter gives a tighter characterization of the optimum error exponent of the channel. For example, for 
the BSC, although the average performance of the linear code ensemble is given by the random coding 
exponent of the Gallager ensemble, the typical performance is given by the expurgated exponent of the 
Gallager ensemble. In this direction, it was also noted recently in |26] that for the 8-PSK Gaussian channel, 
the typical performance of the ensemble of group codes over Z§ equals the expurgated exponent of the 
Gallager ensemble, whereas the typical performance of the ensemble of binary coset codes (under any 
mapping) is bounded away from the same. The second is that in some cases, expurgation may not be 
possible or may not be desirable. For example, (a) in the MAC, the standard expurgation is not possible, 
and (b) if one is looking at the performance of the best linear code for a channel, then expurgation 
destroys the linear structure which is not desirable. In the proposed technique we provide a unified way 
to derive all the three lower bounds on the optimum error exponents, and upper bounds on the ensemble 
average and the typical performance. We wish to note that the bounds derived in this paper are universal 
in nature. The proposed approach appears to be more amenable to generalization to multi-user channels. 
A brief outline of the technique is given as follows. First, for a given constant composition code, we 

^arg and Forney gave only a lower bound in 1131 . 
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define a pair of packing functions that are independent of the channel. For an arbitrary channel, we 
relate the probability of error of a code with a-decoding to its packing functions. Packing functions give 
pair-wise and triple-wise joint-type distributions of the code. This is similar in spirit to the concept of 
distance distribution of the code. Then we do random coding and obtain lower and upper bounds on 
the expected value of the packing functions of the ensemble without interfacing it with the channel. That 
is, these bounds do not depend on the channel. Finally, using the above relation between the packing 
function and the probability of error, we get single-letter expressions for the bounds on the optimum 
error exponents for an arbitrary channel. 

Toward extending this technique to MACs, we follow a three-step approach. We start with a constant 
conditional composition ensemble identical to Il20l . Then, we provide a new packing lemma in which the 
resulting code has better properties in comparison to the packing lemmas in [19] and [20 1. This packing 
lemma is similar to Pokorny's packing lemma, in the sense that the channel conditional distribution does 
not appear in the inequalities. One of the advantages of our methodology is that it enables us to partially 
expurgate some of the codewords and end up with a new code with stronger properties. In particular, we 
do not eliminate pairs of codewords. Rather, we expurgate codewords from only one of the codebooks 
and analyze the performance of the expurgated code. 
Contributions: In summary the key contributions of this work are 

• An exact characterization of the typical error exponent for the constant composition code ensemble 
for the DMC. 

• A new lower bound on the optimum error exponent for the MAC. 

• An upper bound on the average error exponent of the constant composition code ensemble for the 
MAC. 

• A characterization of the typical error exponent for the constant composition code ensemble for the 
MAC. 

This paper is organized as follows: Section [II] introduces terminology, and Section [ill] unifies the 
derivation of all lower bounds on the reliability function for a point-to-point DMC. Our main results 
for the DM-MAC are introduced in Section [TV] Some numerical results are presented in Section [3 and 
Section [VI] concludes the paper. The proofs of some of these results are given in the Appendix. 

II. Preliminaries 

We will follow the notation of 1121 . For any finite alphabet X, let V(X) denote the set of all probability 
distributions on X. For any sequence x £ X n , let P x denote its type. Let Tp denote the type class of 
type P. Let P n (X) denote the set of all types on X. Let T v denote a V-shell, and D(V\\W\P) denote 
conditional I-divergence. In this paper, we consider channels without feedback. 

Definition 1. A discrete memoryless channel (DMC) is defined by a stochastic matrix W : X — > y, where X, 
the input alphabet, and y, the output alphabet, are finite sets. The channel transition probability for n-sequences 
is given by 

n 

W n (y\x) =Hw(yi\xi), 

i=i 

where x = (xi,...,x n ) G X n , y = (yi,...,y n ) e y n . An (n,M) code for a given DMC, W, is a set C = 
{(xj,A) :l<i<M} with (a) x s e X n , D. L c y n and (b) D. L n D. v = for i ^ i'. 
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When message i is transmitted, the conditional probability of error of a code C is given by 

ei(C,W) ^W n {Di\^). 
The average probability of error for this code is defined as 

1 M 

e{C,W)±-Y,ei{C,W). (1) 



M . 

Definition 2. For the DMC, W : X — > y, the average error exponent, at rate R, is defined as: 

Kv (R) = lim sup max - - log e (C, W) , (2) 

w/zere C is tTze set of all codes of length n and rate R. The typical average error exponent of an ensemble C, at rate 
R, is defined as: 

E% (R) = lim inf lim sup max min — — log e(C, W). (3) 

•5^0 n ^oo C:P(C)>1-S C£C Tl 

where P is the uniform distribution over C. 

The typical error exponent is basically the exponent of the average error probability of the worst code 
belonging to the best high probable collection of the ensemble. 

Definition 3. A two-user DM-MAC is defined by a stochastic matrix W : X x y — >■ Z, where X, y, the input 
alphabets, and Z, the output alphabet, are finite sets. The channel transition probability for n-sequences is given 
by 

n 

W"(z|x,y)4 HWMxuVi), (4) 

i=l 

where x = (xi, ...,x n ) G X n , y = (y u ...,y„) £ y n , and z = (zi, ...,z n ) G Z n . 

An (n, M, N) multi-user code for a given MAC, W, is a set C — {(xj,yj, Dij) :l<i<M,l<j< N} with 

. x, G X n , y 3 G y n , C Z n 

. Dij n D V j, = for ^ (i',f). 

When message is transmitted, the conditional probability of error of the two-user code C is given 
by 

eij(C,W)^W n (D^, yj ). (5) 



The average probability of error for the two-user code, C, is defined as 

M N 
i=l j=l 

Definition 4. For the MAC, W : X x y — > Z, the average error exponent at rate pair (Rx,Ry), is defined as: 

E* v (Rx, Ry) — lim sup max - - log e (C, W) , (7) 
where Cm is the set of all codes of length n and rate pair (R x ,Ry)- The typical average error exponent of an 
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ensemble C, at rate pair (Rx,Ry), is defined as: 



Eq V (R x ,Ry) = liminf limsup max min — — loge(C, W), (8) 

<5^° n^oo CcC:F(C)>l-8 CeC Tl 



where P is the uniform distribution over C. 



III. Point to Point: Lower Bounds on reliability function 
A. Packing functions 

Consider the class of DMCs with input alphabet X and output alphabet y. In the following, we 
introduce a unified way to derive all known lower bounds on the reliability function of such a channel. 
We will follow the random coding approach. First, we choose a constant composition code ensemble. 
Then, we define a packing function, tt : C x V(X x X) — > K, on all codebooks in the ensemble. The 
packing function that we use is the average number of codeword pairs sharing a particular joint type, 
V xx . Specifically, for P e V n (X), V xx e V n (X x X), and any code C = {xi, x 2 , Xm} C Tp, the packing 
function is defined as: 

M 

<C, ^x)^EE ^ (x*, x,). (9) 

»=1 jjti 

We call this the first order packing function. Using this packing function, we prove three different packing 
lemmas, each of which shows the existence of a code with some desired properties. 

In the first packing lemma, tight upper and lower bounds on the expectation of the packing function 
over the ensemble are derived. By using this packing lemma, upper and lower bounds on the expectation 
of the average probability of error over the ensemble are derived. These bounds meet for all transmission 
rates below the critical rat^E In the second packing lemma, by using the expectation and the variance 
of the packing function, we prove that for almost all codes in the constant composition code ensemble, 
the bounds in the first packing lemma are still valid. By using this tight bound on the performance of 
almost every code in the ensemble, we provide a tighter bound on the error exponent which we call 
the "typical" random coding bound. As we see later in the paper, the typical random coding bound is 
indeed the typical performance of the constant composition code ensemble. In the third packing lemma, 
we use one of the typical codes and eliminate some of its "bad" codewords. The resulting code satisfies 
some stronger constraints in addition to all the previous properties. By using this packing lemma and 
an efficient decoding rule, we re-derive the well-known expurgated bound. 

To provide upper bounds on the average error exponents, such as those given below in Fact [T] and 
Theorem [TJ for every V xxx &V n (X x X x X), we define a second packing function X : C x V(X x X x 
X) — > R on all codes in the constant composition code ensemble as follows: 

1 M 

HC,V XXX ) ^EEE iT^te^Xfe). (10) 

We call this the second order packing function. As it is clear from the definition, this quantity is the 
average number of codeword triplets sharing a common joint distribution in code C. 

2 This is essentially a re-derivation of the upper and lower bounds on the average probability of error obtained by Gallager in a 
different form. The present results are for constant composition codes. 
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B. Relation between packing function and probability of error 

First, we consider the decoding rule at the receiver, and secondly we relate the average probability of 
error to the packing function. 

Decoding Rule: In our derivation, error probability bounds using maximum-likelihood and minimum- 
entropy decoding rules will be obtained in a unified way. The reason is that both can be given in terms 
of a real-valued function on the set of distributions on X x y. This type of decoding rule was introduced 
in |25J as the a — decoding rule. For a given real-valued function a, a given code C, and for a received 
sequence y € y n , the a — decoder accepts the codeword x G C for which the joint type of x and y 
minimizes the function a, i.e., the decoder accepts x if 

x = argmina(P • Vy| x ). (11) 

It was shown in [25] that for fixed composition codes, maximum-likelihood and minimum-entropy are 
special cases of this decoding rule. In particular, for maximum-likelihood decoding, 

a(P-V)=D(V\\W\P) + H{V\P), (12) 

and for minimum entropy decoding, 

a(P-V)=H(V\P), (13) 

where P is the fixed composition of the codebook, and V is the conditional type of y given x. 
Relation between probability of error and packing function: Next, for a given channel, we derive an 
upper bound and a lower bound on the average probability of error of an arbitrary constant composition 
code in terms of its first order and second order packing functions. The rest of the paper is built on this 
crucial derivation. Consider the following argument about the average probability of error of a code C 
used on a channel W. 

M M 

e(C, W) = —Y J W n (Dt\ Xi ) = — £ W n ({y : a(P ■ V y[xi ) > a(P ■ V y[X] ) for some j + i\ \m) 



M ^ v 11 ' M 
i=i i=i 



/ 2 -nlDCVY\x\\W\P)+Hv(Y\X)] 



1 M 



i=l 



(14) 



where V„ and A{ (V XXY ,C) are defined as follows 

K = {Vxxy e Pn(* x X x y) : V x = V x = P , a(P ■ V Y{X ) < a(P, V Y \ X )} , (15) 
A i ( v xxy' C ) ~\{y : ( x ^ x i-y) G T v XXY for some (16) 

From the inclusion-exclusion principle, it follows that A i (V XXY ,C) satisfies 

B t {V XXY ,C) - C t {V XXY ,C) < MV XXY ,C) < Bi<y xjtY ,C)> (17) 

where 

Bi(V xjtY ,C) =ElT^(xi,x,-)|{y : y G T Vyix ^,x 3 )}\, (18) 
CfeC) = ^E 1 Tv xje (xi,x i )lT Vx;e (x i ,x fc )|{ y :yeT Vy|xjt (x i ,x i )nT Vylxi ( Xi ,x fc )}|. (19) 
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Next, we provide an upper bound on the second term on the right hand side of (fT4t as follows. 

M _ M 



g E ^ ( v xxy> C) < ^ X> ^ V xxy^) 

i=l i=l 
1 M 

= mEE (x* , xj ■ ) | {y : y 6 Ty y |xj . (x* , x, ) } 



M 
»=1 j^i 



1 



On the other hand 



so we can conclude that 



= n(C,V X x)2 nH ^ x ^ 
{y : (xi,Xj,y) e TV XJfy for some j ^ i} C r Vy|X (xi), 



1 M 



(20a) 

(20b) 

(20c) 
(20d) 

(21) 
(22) 



Combining the above with lfl4|l , we have an upper bound on the probability of error fn terms of the first 
order packing function as follows. 



e(C,W)< E 2-"[^i-H w l p )]min{2-" 7 -^ Ay l x )7r(C,^),l} 



(23) 



Next, we consider the lower bound. For that, we provide a lower bound on Bi and upper bound on 
Ci as follows. 



M 



M E B^xxy'C) = E E l ^ xx (Xl ' x ^ I {y : y G Tv v\xx ( x *' x i)i 

i=i 

>7r(C,V xjf )2 n l ff ^*)- i l J 



(24) 



and 



»=i 

1 M — « 

= M E E E lj V xA (x*. *j)l3V XJt x fc) I {y : y 6 IV y|xJt (x l7 Xj ) D JV y|xx (xi, x fe )} 
i=l j^i k^i,j 

— , 1 M _ 

E i7 EE E KM (x " x f Xt) |{ y:yer w (x " x f xt) } 



*XXXY ■ 



i=1 j^i k^i,j 



(25) 



^ XXXI" : 
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Combining fl4t , l|24t , and J23t we have the following lower bound on the average probability of error. 



e(C,W)> T 



■n[D{V Y \x\\W\P)+Iv{XAY\X)+8} 



(26) 

Observe that these upper and lower bounds apply for every code C. We have accomplished the task of 
relating the average probability of error to the two packing functions. The key results of this subsection 
are given by ((23) and ((26). Next we use the packing lemmas to derive the bounds on the error exponents. 

C. Random Coding Packing Lemmas 

Lemma 1. (Random Coding Packing Lemma) Fix R > 0, S > 0, a sufficient large n and any type P of 
sequences in X n satisfying H{P) > R. For any V xx e V n {X x X), the expectation of the first order packing 
function over the constant composition code ensemble is bounded by 

2 n(R-i v (XAX)-5) <E(n(X M ,V xx )) < 2 »(*-MXAX)+tf) ) (27) 

where X M = (Xi,X 2 , ...,Xm) C Tp are independent and XiS are uniformly distributed on Tp, and 2 n ( R ~ 5 ^ < 
M < 2 nR . Moreover, the following inequality holds for the second order packing function: 

E (\{X M ,V XXX )) < 2«[ 2 «-Mxax)-mxaxx)+45] j Qr aU y^^ e v ^ x xXxX y (28) 

Proof: The proof follows directly from the fact that two words drawn independently from Tp have 
a joint type V xx with probability close to 2~ nI ( x,xX K The details are provided in the Appendix. ■ 

Lemma 2. (Typical Random Code Packing Lemma) Fix R > 0, 5 > 0, a sufficient large n and any type P 
of sequences in X n satisfying H(P) > R. Almost every code, C l , with 2"(- R_l5 ) < M < 2 nR codewords, in the 
constant composition code ensemble satisfies the following inequalities 

2 n[R-I V (XAX)-25] < ^ C \V XX ) < 2"[ R -M*AX)+2«5] ^ fl „ y^ g p ^ x x ^ ^ 

and 

X (C\V XXX ) < 2«[ 2 «~Mxax)-mxaxx)+45] j Qr aU y x gg e -p n (x x X x X). (30) 

Proof: The proof is provided in the Appendix. In the proof, we evaluate the variance of the packing 
function and use Chebyshev's inequality to show that with high probability the packing function is close 
to its expected value. ■ 

Lemma 3. (Expurgated Packing Lemma) For every sufficiently large n, every R > 0, 8 > and every type P 
of sequences in X n satisfying H(P) > R , there exists a set of codewords C ex — {xi,x 2 , ...,xm*} C Tp with 
M* > 2 " ( ^~' 5) , such that for any V xx e V n (X x X), 

n(C ex ,V xx ) < 2"(«- f v-(XAl)+25) i (31) 

and for every sequence Xj e C ex , 

\T V . (xi) n C ex \ < 2 n ( R -^<- XAX )+ 2S \ (32) 
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Proof: The Proof is provided in the Appendix. The basic idea of the proof is simple. From Lemma 1 
we know that for every V xx , there exists a code whose packing function is upper bounded by a number 
that is close to 2 n ( R ~ Iv ( XA,x ^ . Since the packing function is an average over all codewords in the code, we 
infer that for at least half of the codewords, the corresponding property ll32t is satisfied. In the Appendix, 
we show that there exists a single code that works for every joint type. ■ 

D. Error Exponent Bounds 

Now, we obtain the bounds on the error exponents using the results from the previous three subsections. 
We present three lower bounds and two upper bounds. The lower bounds are the random coding 
exponent, typical random coding exponent and expurgated exponent. All the three lower bounds are 
expressed as minimization of the same objective function under different constraint sets. Similar structure 
is manifested in the case of upper bounds. For completeness, we first rederive the well-known result of 
random coding exponent. 

Fact 1. (Random Coding Bound) For every type P of sequences in X n and < R < H(P), S > 0, every 
DMC, W : X — > y, and 2 n ( R ~ 5 '> < M < 2 nR , the expectation of the average error probability over the constant 
composition code ensemble with M codewords of type P, can be bounded by 

2-n[E rL (R,P,W)+38] < p £ < 2-«[ B r(-K,-P,W0-2<5] (33) 

whenever n > m(\X\, \y\,S), where 

EJR,P,W) = min D(V Ylx \\W\P) + \I V (X A XY) - R\ + . (34) 
E rL (R,P,W) = min D(V Y \ X \\W\P) + I V (X A XY) - R, (35) 



I V {X/\XY)>R 



and 



V T = {Vxxy eV(XxXxy):V x = V x =P, a(P,V Y , x ) < a(P, V Y]X )}. (36) 



In particular, there exists a set of codewords C r — {xi,x 2 , ...,xm} C Tp, with M > 2 n ^ R 5 \ such that for every 
DMC, W :X -+y, 

e (C r ,W)<2- n ^ R > p > w *>- 35 l (37) 

Proof: The proof is straightforward and is outlined in the Appendix. ■ 
It is well known that for R > R cr it, the random coding error exponent is equal to the sphere packing 
error exponent, and as a result the random coding bound is a tight bound. In addition, the following is 
true. 



Corollary 1. For any R < R c 



max E rL (R,P,W) = max EJR.P.W). (38) 



Proof: The proof is provided in the Appendix. ■ 
Next we have an exact characterization of the typical performance of the constant composition code 
ensemble. 
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Theorem 1. (Typical random Coding Bound) For every type P of sequences in X n , 6 > 0, and every 
transmission rate satisfying < R < H{P), almost all codes, C* = {xi, X2, xm} with Xi e Tp for all i, 
M > 2 niR - s \ satisfy 

2 -n[E TL {B.,P,W)+4S] < g /W w \ < 2 -nlE T (R,P,W)-3d]^ (39) 

for every DMC, W : X — >• y, whenever n > n\{\X\, \y\, S). Here, 

E T (R,P,W) = min D{V Y \ X \\W\P) + \I V {X A XY) - R\ + , (40) 

V XXY^ 

E TL {R,P,W) = min D(V Y \x\\W\P) + I V [X A XY) - R, (41) 

V XXY^ Vt - 

I V {X/\XY)>R 

where 

T >t = {V xiY ^V{Xy.Xy.y): V x = V x = P, Iv{X A X) < 2R , a(P, V Y]X ) < a{P, V Y{X )}. (42) 

Proof: The proof is provided in the Appendix. ■ 
In Theorem [TJ we proved the existence of a high probability (almost 1) collection of codes such that 
every code in this collection satisfies (39). This provides a lower bound on the typical average error 
exponent for the constant composition code ensemble as defined in Definition [2] In the following, we 
show that the typical performance of the best high-probability collection cannot be better than that given 
in Theorem [T] 

Corollary 2. For every type P of sequences in X n , 8 > 0, and every transmission rate satisfying < R < H{P), 

E T (R, P, W) < E T av (R) < E TL (R, P, W) , (43) 

for the constant composition code ensemble. 

Proof. The proof is provided in the Appendix. ■ 
Clearly, since the random coding bound is tight for R > R cr it, the same is true for the typical random 
coding bound. For R < R cr . L t we have the following result. 



Corollary 3. For any R < R c 



max E TL (R,P,W) = max E T (R,P,W). (44) 

pgv(x) Pev(x) 



Proof: The proof is very similar to that of Corollary [T] and is omitted. ■ 
It can be seen that the typical random coding bound is the true error exponent for almost all codes, 
with M codewords, in the constant composition code ensemble. A similar lower bound on the typical 
random coding bound was derived by Barg and Forney [13| for the binary symmetric channel. Although 
the approach used here is completely different from the one in 1131 . in the following corollary we show 
that these two bounds coincide for binary symmetric channels. 

Corollary 4. For a binary symmetric channel with crossover probability p, and for < R < R cr it 

E T {R,P,W) = E TRC (R), (45) 
where Etrc is the lower bound for the error exponent of a typical random code in [13 J. 
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Finally, we re-derive the well-known expurgated error exponent in a rather straightforward way. 

Fact 2. (Expurgated Bound) For every type P of sequences in X n and < R < H(P), 5 > 0, there exists a set 
of codewords C ex = {xx,x 2 , ...,x M »} c T P with M* > 2 " ( ^ - , such that for every DMC, W : X ->• y, 

e{C ex ,W) < 2-"[- E ex(fl,P,w)-35] ^ 

whenever n > n\{\X\, \y\,8), where 

E ex (R,P,W) = min D(V Y \ X \\W\P) + \I V (X A XY) - R\ + (47) 

where 

V ex = {Vxxy e V{X x X x y) : V x = V jt =P ) I V {X A X) < R , a(P, V Y[X ) < a(P, V Y \ X )} (48) 

Proof: The proof is provided in the Appendix. ■ 
Note that none of the mentioned three bounds have their "traditional format" as found in 1121 , Q, 
but rather the format introduced in |25ll by Csiszar and Korner. It was shown in |25| that the new 
random coding bound is equivalent to the original one for maximum likelihood and minimum entropy 
decoding rule. Furthermore, the new format for the expurgated bound is equivalent to the traditional 
one for maximum likelihood-decoding and it results in a bound that is the maximum of the traditional 
expurgated and random coding bounds. 

IV. MAC: Lower Bounds on reliability function 

Consider a DM-MAC, W, with input alphabets X and y, and output alphabet Z. In this section, we 
present three achievable lower bounds on the reliability function (upper bound on the average error 
probability) for this channel. The method we are using is very similar to the point-to-point case. Again, 
the goal is first proving the existence of a good code and then analyzing its performance. The first step 
is choosing the ensemble. The ensemble, C, we are using is similar to the ensemble in [20J. For a fixed 
distribution, PijPx\uPy\u> the codewords of each code in the ensemble are chosen from Tp x . u (u) and 
Tp Y{u (u) for some sequence u e T Pu . Intuitively, we expect that the codewords in a "good" code must 
be far from each other. In accordance with the ideas of Csiszar and Korner 1121 , we use conditional 
types to quantify this statement. We select a prescribed number of sequences in X n and y n so that the 
shells around each pair have small intersections with the shells around other sequences. In general, two 
types of packing lemmas have been studied in the literature based on whether the shells are defined on 
the channel input space or channel output space. The packing lemma in |19] belongs to the first type, 
and the one in (20ll belongs to the second type. All the inequalities in the first type depend only on 
the channel input sequences. However, in the second type, the lemma incorporates the channel output 
into the packing inequalities. In this work, we use the first type. In the following, we follow a four step 
procedure to arrive at the error exponent bounds. In step one, we define first-order and second-order 
packing functions. These functions are independent of the channel statistics. Next, in step two, for any 
constant composition code and any DM-MAC, we provide upper and lower bounds on the probability 
of decoding error in terms of these packing functions. In step three, by using a random coding argument 
on the constant composition code ensemble, we show the existence of codes whose packing functions 
satisfy certain conditions. Finally, in step four, by connecting the results in step two and three, we provide 
lower and upper bounds on the error exponents. Our results include a new tighter lower bound on the 
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error exponent for DM-MAC using a new partial expurgation method for multi-user codes. We also give 
a tight characterization of the typical performance of the constant composition code ensemble. Both the 
expurgated bound as well as the typical bound outperform the random coding bound of [20J, which is 
derived as special case of our methodology. 

A. Definition of Packing Functions 

Let Cx — {xi, x 2 , xm x } and Cy — {yi,y2> — jMy} be constant composition codebooks with jq € 
Tp x (u) and y 3 e Tp Y{u (u), for some u e Tp u . In the following, for a two-user code C = Cx x Cy, we 
define the following quantities that we will use later in this section. 

Definition 5. Fix a finite set U, and a joint type V UXYXY € V n (H X (X x 3^) 2 ). For code C, the first-order 
packing functions are defined as follows: 

1 M x My 

Nu(C,V UX y) '^EE^Kx.y,), (49a) 

i=i j=i 

M x M Y 

N X {c,v UXYji ) ^EEE\„,(^.y^). < 49b > 

A r i=l 3=1 fc^j 

w Wr) = E E E i*w < u > x - (49c) 

i=i j=i i^j 

M x My 

V UXYXY ) = — — E E E E 1*W„ x„ y„ x fc , yi ). (49d) 
i=l j=i fc^i j^j 

Moreover, for any V UXYXYXY £ V n (14 x (X x y) 3 ), we define a set of second-order packing functions as 
follows: 

A *( C > V uxyxx) = Jj^YsY, E ^^Kxi.y^Xfe.XfeO, (50a) 

«,j fe#i k'=£i,k 

Ay(C,V uxyyy ) ±——Y^Y. E !t w? . (^x^y^y^y^), (50b) 
Axy(C,V uxyxyxy ) = M * M E E E iTv^jf^Kx^y^Xfe^^Xfe^y^). (50c) 

i,j fe#i k'^i,k 

The second-order packing functions are used to prove the tightness of the results of Theorem [2] and 
Theorem [3] Next we will obtain upper and lower bounds on the probability of decoding error for an 
arbitrary two-user code that depend on its packing functions defined above. 

B. Relation between probability of error and packing functions 

Consider the multiuser code C as defined above, and a function a : V(U x X x y x Z) — > K. Taking 
into account the given u, a-decoding yields the decoding sets 

D iS = {z : n.:/',, x , y z i < a(P u , Xfc , yi , z ) for all (k,l) ^ . (51) 
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The average error probability of this multiuser code on DM-MAC W, can be written as 



(52) 



M X M Y ^ w J J M X M Y ^ ^ 1 1JJ ' M X M Y 

i,j kjti i,j 1+3 l,j k+i 

m 



The first term on the right side of l l52l can be written as 
i—^^(|J^|x l ,y,) 



M X M Y . 

i,3 k+i 



= M l M E ^"(i 2 : a(-Pu,x fc: y J: z) < a(P u , X! , yj , z ),for some k + i] |u,Xi,y^ 

X Y i,3 

= ^ T \ 7 y y ^(ziu.x^y.) 

M X M Y 4-? ^ 1 1 ww 

M a(P u , 5Cfc , yj ., z )<a(P u , Xi , y3 , z ) 
for some k^i 

= M^E E E l 2 v pjty ^(u > x il y J - J x fc> z)W»(z|u > x i ,y i ) 

for some k^i 

= E r . [ ] ^j;i^(u,x i ,y j )4( tol O 

(53) 

where 

A ?,j ( v uxyxzi G ) ~ K z : (u,Xi,yj>x fe ,z) e Tv^^for some fc ^ i}| 

V*,« = {^t/xyxz : a(V UXYZ ) > a(V UXYZ ), V ux = V ux = P UX ,V UY = P UY ] . (54) 

Note that V r x is a set of types of resolution n, therefore, we use a subscript n to define it. Similarly, the 
second and third term term on the right side of J52l can be written as follows: 



i— ^"(IjA^y;) 



M X M Y . 

i,3 1+3 

= E r ^ n[D(Vz ^ mVxYU)+ ^ 

^UXYYZ^Y,n l >-? 

(55) 

where 



A L ( v uxyyz? c ) - K z : (u,Xi,yj,yj,z) e 7V uxy?z for some i ^ j}| 
^ r Vxrrz • ^vuxyz/ ^ ^"uxYZh y ux - * ux, *uy — * UY 



Vv„ = {Vrfz : Q (Wz) > a(V UXYZ ), V ux = Pux,V UY = V UY = P UY } , (56) 
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and, 

l, J k=jii 
1+3 

V o-n\D(V zlxvu \\W\V XYU ) + H v (Z\XYU)\ \ 1 ^. / A ^Xy (-is ' 

Z [jlf x My^ ""^ ' J:) ' ,J \ V UXYXYZ^> 

^UXYXYZ^XY.n l J 

(57) 

where 

A ?J ( v uxyxyz> c ) ~ K z : (u> x i>yj. x fc.y*> z ) e \ xyX?z for some fc ^ i,Z ^ j}| 

V^y,„ ± {V UXYXYZ : «(feyz) > a(V UXYZ ), V ux = V ux = P UX ,V UY = - P^} . (58) 

Clearly, Af - (V UXYXZ ) satisfies 

Bfj (Vuxyxzi - Cfj (V UXYXZ , C) < Afj (V UXYXZ , C) < Bfj (V UXYXZ , C) , (59) 

where 

B i, 3 ( v uxyxz' C ) - J2 1t v uxyx ( U , x ^y^ x fc)-K z : z e T v z|[/X ^(u,x 4 , yj ,x fc }|, (60) 



(WxZ- C )~E E lTv ux Y x (U ' Xl ' y J' "^^xrt Xl ' Xfc ') 

• |{z : z e T Vz|uxrJf (u,x i ,y j ,x fe )nTv z|j7xy ^(u,x i ,y j ,x fe /)}|. (61) 

Having related the probability of error and the function Bf^ Bj 3 and BfJ, our next task is to provide 
a simple upper bound on these functions. This is done as follows. 

E 1t Wxy ( u ' *i>yj) B & ( V uxyxz, C) 

i,3 

= ^EE 1 \^( u ' x «-yi-^ |{ z : z e T ^,™( u > x -y;> x fc)}| 

= 2 „M(z|,xyx )jV;f(C)W) (62) 

Similarly, we can provide upper bounds for Bjj and BfJ. Moreover, we can also provide trivial upper 
bounds on A(-) functions as was done in the point-to-point case. 
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The same bound applies to A Y and A XY . Collecting all these results, we provide the following upper 
bound on the probability of error. 

e(C,W) < ]T 2-"^i^ll w l^)Uin{2-" / -^ AZ l xyc/ )7V x (C,y [/xy ^),l} 

Vuxyxz 

+ J- 2-"^(^i^ll^l^^)]min{2-" / -( fAZ l^iV y (C,W ? .),l} 

VlJXYYZ 

+ £ 2- n W v zi^ w \ v ^mm{2- nI ^* Y ^ XYU ^ (63) 



~^XY. 



UXYXYZ 



Next, we consider lower bounds on B(-) functions and upper bounds on C( ) functions. One can use 
a similar argument to show the following 



M X M- 



Kr E ^uxy (»• W)!® (V uxyjtz , C) > 2 nW(ZWXYX ) - S]Nx{ ^ V[ 



uxyxi- 



Similar lower bounds can be obtained for B Y and B XY . Moreover, we have the following arguments for 
bounding from above the function C x . 

MxM Y E 1t v uxy ( u ' x *' Vi) ■ C ?j ( v uxyxz) 

= m^EK^K^OE Y, liV t , xyJt (u,x i ,y J -,x fc )l7v t , jryJt (u,x i ,y J -,x fc 

i,j k^i k'^k,i 

' |{ z : z e T ^ z|[/X ^(u,x 4 ,y,,x fe ) nTy z|Lrx ^(u,x 4 ,y,,x fe 0} 

= m^;E E E E_ 1 ^ X v Jt *( u '^' y ^ Xfc ' x ^)|{ z:zeT wv**( u '^y^ x *' x *')}| 

^(/xyxxz* fc^t k'^k.i 
xy x z~^u xy x z 

s E E 1 '-v„ y „(u,x i ,y J ,x l ,x ( ,) 

xy xx z m k>¥^i & 7^)^ 

E 2«*(^™*)A x (C, W ^). (64) 



Similar relation can be obtained that relate C Y and Ay, C xy and Axy- Combining the lower bounds on 
S( )-functions and upper bounds on C( )-f unctions, we have the following lower bound on the probability 
of decoding error. 
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e(C,W) 



> 



E 2- 



i[D(V z[XYU \\W\V XYU )+I v (XAZ\XYU)+S] 



+ j2 2 ~ n[D 



(V zlXYU \\W\V XYU )+I v (Y/\Z\XYU)+8] 



V UX YY Z 



E 



N x 



Ny 



1 [/xyxxz' 



^ 2 n/(yAZ|rayy) Ay 



2-™[n(V Z |xyi7l|W|yxy(7)+/v(XyAZ|XyC/)+«5] 



N 



XY 



E 

^uxyxxyyz- 



2nI(XY/\Z\UXYXY) ^ 



XY 



(65) 



This completes our task of relating the average probability of error of any code C in terms of the 
first and the second order packing functions. We next proceed toward obtaining lower bounds on the 
error exponents. The expressions for the error exponents that we derive are conceptually very similar 
to those derived for the point-to-point channels. However, since we have to deal with a bigger class of 
error events, the expressions for the error exponents become longer. To state our results concisely, in the 
next subsection, we define certain functions of information quantities and transmission rates. We will 
express our results in terms of these functions. The reader can skip this subsection, and move to the next 
subsection without losing the flow of the exposition. The reader can come back to it when we refer to it 
in the subsequent discussions. 

C. Definition of Information Functions 

In the following, we consider five definitions which are mainly used for conciseness. 

Definition 6. For any fix rate pair R x ,Ry > , and any distribution V UXYXY 

Fu(V UX y) = I(X AY\U), 
F x(V UXYX ) = I(X AY\U)+ I V (X A XY\U) - R x , 
f y(V uxyy ) = I(X AY\U) + I(Y A XY\U) - R Y , 

,Y\U) + I(XY AXY\U)-R X -Ry 



Fxy{V uxyxy ) 4 I(X A Y\U) + I(X A Y\U) 
Moreover, for any V UXYXYXY e V (U x (X x y) 3 ), we define 

E s(Vuxyxx 
E s ( v uxyyy) 

pXY / v \ A 

\ V UXYXYXY> 



, , eV(Ux(Xx y) 2 ), we define 

(66a) 
(66b) 
(66c) 
(66d) 



= I(X A XYX\U) + I(X A XY\U) + I(X A Y\U) - 2R X 
I(Y A XYY\U) + I(Y A XY\U) + I(X A Y\U) - 2R Y , 

\ A 



(67a) 
(67b) 



I(XY A XYXY\U) + I(XY A XY\U) + I(X A Y\U) + I(X A Y\U) + I(X A Y\U) - 2R X - 2R Y . (67c) 
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'xyu EV (X x y x U), we define the sets of distributions 
Vx = {V UXY xz ■ aiVuxvz) > a(V UXYZ ),V ux = V ux = P t 



Definition 7. For any given R x , Ry > 0, P; 
and V r XY us follows. 



ux = ^uxiVuy = Puy} , 
uxyyz '■ &{Vuxyz) > Oi{V UXYZ ), V ux = Pu : 
'uxyxyz '■ a ( v uxYz) > a(V UXYZ ),V ux = 
Y L and V XY are seis °f distributions and defined i 
n L = {Vuxyxz e Vx ■ UX A XYZ\U) > R x ) , 
V r Y L = {Vuxyyz e V Y ■ I(Y A XYZ\U) > R Y ) , 
Vx Y — j^r " ~ ,; ' T> vv 7 '■ ■ - L • 



: v uy - p uy} , 
Vux ~ Pux , Vuy — V UY — Puy } ■ 

Moreover, V X L , V r Y L and V r XY 



^UXYYZ 
^UXYXYZ ^ V* 



V XY ■ HXY A XYZ\U) + I (X AY) > Rx + R Y } ■ 

Definition 8. For any given R X ,R Y > 0, Pxyu <eV (X x y x U), we define the sets of distributions 
and V XY as follows 



v x — 



VuXYX 



VuXYY 



v X y — ' 



^uxyxy • 



Vxu — V xu ~ Pxu, Vyu — Pyu 
Fu(Vuxy),Fu(V uxy ) <R x +Ry 
F x(V UXYX ) < Rx + Ry 
a(V UX Yz) > a(V UXYZ ) 

Vxu — Pxu, Vyu — V YU = Pyu 
Fu(Vuxy),Fu(V uxy ) <R x +Ry 
Fy(V uxyy ) < Rx + Ry 
a(VuxYz) > Oi(V UXYZ ) 



V x , V Y 

(68a) 
(68b) 
(68c) 

(69a) 
(69b) 
(69c) 

vl vl, 



(70a) 



(70b) 



Fu(V L 



Vxu = V xu = P X u, Vyu = V YU = Pyu 
r uxv), Fu(V UXY ), Fu(V UXY ), Fu(V UXY ) < Rx + Ry 
F x(V UXYX ),F x (V UXYX ) < Rx + Ry 
F Y (V V - 



v uxyy)' f y(V uxyy ) < Rx + Ry 
f xy(V uxyxy ),F X y(V uxyxy ) < Rx + Ry 
a(VuxYz) > a(V v 



-Vuxyz) 

T L T L T L 

Moreover, V x , V Y , and V XY are seis °f distributions and defined 
Vl' L = lv„ „,„-,„ F Vl : Kit A XYZ\U) > R Y ) 



(70c) 



' as 



v x ={V UXYXZ EV T X :I(XAXYZ\U)>R X ), 
Vy L = {Vuxyyz e Vl : I(Y A XYZ\U) > R Y ] , 
Vxy = {Vuxyxyz e V T XY : I(XY A XYZ\U) + I(X A Y) > R x + R Y } ■ 
Definition 9. For any given R X ,R Y > 0, Pxyu <eV (X x y x U), we define the sets of distributions 



(71a) 
(71b) 
(71c) 

Vex \)ex 
X > V Y ' 
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and V XY as follows 



Vex A 
X — 



V 



UXYX 



Vxu = Vjcu ~ Pxu, Vyu — Pyu 
Fu(V U xy),F u {V uxy ) < mm{Rx,R Y } 
F X (V UXYX ) <mm{R x ,R Y } 
a{V UX Yz) > a{V UXYZ ) 



(72a) 



yex A 



UXYY 



Vxu = Pxu,V YU = V Y 



YU - r yu 



(72b) 



V, 



UXYXY ■ 



Fu{V UX y),Fu(V uxy ) < mm{R x ,R Y } 
fy(Vf) ^ mm{R x ,R Y } 
a(V UX Yz) > ot{V UXYZ ) 

Vxu — V X u ~ Pxu, Vyu = V YU = Pyu 
F u (Vuxy),F u (V U xy),Fu(V U xy),Fu(V UX y) < mm{R x ,R Y } 
Fx{V UXYX ),F x {V UXYX ) < mm{R x ,R Y } 

Fy(V UX yy)> f y(Vuxyy) < mm{R x ,R Y } 
Fxy{V uxyxy ),F xy {V uxyxy ) < mm{R x ,R Y } 

a(V UX Yz) > a(V UXYZ ) 

Definition 10. For any given R x ,Ry > 0, P X yu e V (X x y x U), and V UXYXY e V (U x (X x y) 2 ), we 
define the following quantities 



Vex A 
XY — 



(72c) 



E x (Rx,Ry, W, P X yu, V uxyx ) 4 D(V z \xyu\\W\Vxyu) + Iv(X A Y\U) + \I(X A XYZ\U) - R x \+, 



(73a) 



Ey{Rx,Ry,W,P X yu,V UXY y) — F ) {Vz\xyu \\W\Vxyu) + Iv(XA Y\U) + \I(Y A XYZ\U) - R Y \ + , 

(73b) 

Exy{Rx,Ry, W, Pxyu, Vjjxyxy) — 

D(V ZI xyu\\W\Vxyu) + Iv(X A Y\U) + \I(XY A XYZ\U) + I V (X A Y\U) - R x - R Y \ + . (73c) 

Moreover, we define 

E l x (Rx,Ry, W, Pxyu, V uxyx ) = D(V z \xyu\\W\V X yu) + W{X A Y\U) + I(X A XYZ\U) - R x , (74a) 
E^(R x ,Ry, W, Pxyu, V UXY y) = D(V z \xyu\\W\V X yu) + W{X A Y\U) + I(Y A XYZ\U) - R Y , (74b) 

E X y(Rx , Ry ,W, Pxyu ,V UXY xy) — 

D(V z \xyu\\W\Vxyu) +Iv(XA Y\U) + I(XY A XYZ\U) + I V (X A Y\U) - R x - Ry, (74c) 

and, 

E%{R x ,Ry,W,Pxyu,V%)^ min a E^R x ,R Y ,W,P XY u,V UXY ^, 

^rryvfl?^ ^ 'ft 



E«' l (R x ,Ry,W,Pxyu,V%)^ min E^(R x ,R y ,W,Pxyu,V uxy s), 

V UXY(3Z fe k /3 



(75a) 
(75b) 



/or a e {r, T, ex}, and (3 G {X, y, IF}. 
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D. Packing Lemmas 

As we did in the point-to-point case, here we perform random coding and derive bounds on the 
packing functions. The results will be stated as three lemmas, one for the average and one for the typical 
performance of the ensemble, and finally one for the expurgated ensemble. These results will be used in 
conjunction with the relation between the packing functions and the probability of error established in 
Section IIV-BI to obtain the bounds on the error exponents. 

Lemma 4. Fix a finite set U, P XY u 6 V n (X x y x U) such that X - U - Y, R x > 0, R Y > , S > 0, 
2 n(Rx-s) < Mx < 2 nR x , 2™(^- 5 ) < M Y < 2 nRr , and u e T Pu . Let X Mx = {X 1 ,X 2 ,...,X Mx } and 
Y My = {Yi, Y 2 , Ym y } are independent, and XiS and YjS are uniformly distributed over Tp x{u (u) and Tp Ylu (u) 
respectively. For every joint type V UXYXY 6 P n (U x (X x y) 2 ), the expectation of the packing functions over the 
random code X Mx x Y My are bounded by 



2-n[F u (V UXY )+5] < 
2 -n[F x {V UXYX )+38] < j 
2 -n[F Y (V UXY9 )+38] < j 
2~n[F X Y(V UXYXY )+4:S] < jg 



Nu(X Mx xY M *,V UXY ) 
N x (X Mx xY My ,V uxyx ) 
N Y (X Mx x Y MY ,V t 



UXYY . 



< 2-n[F v (Vuxy)-28] 
< 2 -n[F x (V UXYX )-4S]^ 
< 2 -n[F Y (V UXYY )-4S]^ 



N XY (X Mx xY My ,V uxyxy ) 



< 2 -™[Fx Y (V UXYXY )-4:5] 



whenever n> n (\U\,\X\,\y\,5). Moreover, for any V l 



UXYXYXY 



S V n {U x {X x y) 3 ) 



E 



M Y 



UXYXX 



v) 



E 



M X ^ vMy T/ 

* T 



a X y(x Mx x y 



UXYYY 
■ ' / VVA > X?) 



< 2 -«(^(V„ XYM )-4i) i 

< 2 -™{e^ (V UXYYY )-U) 



(76a) 
(76b) 
(76c) 
(76d) 

(77a) 
(77b) 
(77c) 



whenever n > n (\U\, \X\, \y\, S). 

Proof: The proof is provided in the Appendix. ■ 

Lemma 5. Fix a finite set U, P X yu 6 V„{X X y X W) smc/i f/wf X - J7 - F, i? x > 0, R Y > , <S > 0, 
2 n(R x -8) < Mjc < 2 ™ fl ^, 2™( i? ^-' 5 ) < Afy < 2 nRy , and ue%. A/mosf every multi-user code C = C x x Cy, 
C x = {xi,x 2 , ...,xm x } C Tp X|t; (u) and C Y = {yi,y2, •••,yM y } C T Pyiu (u), in the constant composition code 
ensemble, C, satisfies the following inequalities: 



2 -nlF u (V UXY )+38] < Nu(C,Vuxy) < 2^ n ^ Fu< ^ VuxY ^ 3S \ 
2 -nlF x (V UXYX )+h8] < jVx(C, V^^y^) < 2~ n[Fx(V f*™ ) ~ 5l5] , 
2-n[Fy(V c;xr? )+55] < jVy(C, V ax yy) < 2~ n[FYiy " XYXY) ~ bS \ 
2 -n[F XY {V UXYXY )+h8] < 7V X y(C, V^y^y) < 2~™ [Fxy (Vyxy)-^] ; 



(78a) 
(78b) 
(78c) 
(78d) 
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for all V UXYXY e V n {U X (X X y) 2 ), and 

Ax(C,V UXYXX ) < 2- n ( E S ( - v ux Y xx)-55) 7 (79a) 

A Y (C,V UXYYY ) < 2-"( E s(v uxy v*-)-5«) 5 (79 b ) 

A ^( C 'WxKf) < 2""K y (W«)-™). ( 79c) 
/or aZZ ^axrxrxr e ^« (li x (X x y) 3 ), whenever n > n (\U\, \X\, \y\,S). 

Proof: The proof is provided in the Appendix. ■ 

Lemma 6. For every finite set U, P X yu G P^x^x W) suc/j f/wf X - U -Y, R x > 0, R Y > , 

5 > 0, and u e Tp u , there exist a multi-user code C* = C* x x C Y , C* x — {xi,X2, ...,xm* } C Tp X|!7 (u) and 
Cy = {yi,y2, ...jm* } C Tp Y]u (u) with M* x > 2 " * ' , M Y > 2 " ( " 2 y - , such that for every joint type 
Vuxyxy eV n (Ux(Xx y) 2 ), 

Nu{C*,V UXY ) < 2- n ^ Vvx ^~^ (80a) 

N X (C*, V UXYX ) < 2- n ^ < - v ux Y x)-es] (80b ) 

N Y (C*, V UXYY ) < 2- n ^ ( - v uxYY)~es] (80c ) 

N XY (C*,V UXYXY ) < 2-^ Fxy1 - v uxyxy)-^] (80d) 
and for any 1 < i < M x , and any 1 < j < M Y , 

l TvuxY (u,^,y 3 ) < 2 -«[^(W)-ndn{Jl ac ,Jl jr }-66] (g la) 

^l Tv . .(u,X i ,y J ,Xfc) < 2-™[^(^ i -x)-n»«{«x,^}-6,5] ( glb ) 

^2l TVuxY9 (u,ya, yj ,yi) < 2-»[^(v PXV? )-min{flx,iir}-65] (81c) 

VV% (u,x 4 ,y„x fc ,y ; ) < 2-»[ F **<W**)-™{K*,Hv}-6«] ( 8 ld) 



[/xyxy 



whenever 



n>n (\U\,\X\,\y\,5). 



Proof: The proof is provided in the Appendix. ■ 
As it is shown in the Appendix, the above property is derived by the method of expurgation. Unlike 
the point-to-point case, expurgation in the MAC is not a trivial procedure. To see that, observe that 
expurgating bad pairs of codewords results in a code with correlated messages, which is hard to analyze. 
Instead, what we do is a sort of "partial" expurgation. Roughly speaking, we start with a code whose 
existence is proved in Lemma H] and eliminate some of the bad codewords from the code with the larger 
rate (as opposed to codeword pairs). By doing that, all messages in the new code are independent, and 
such a code is easier to analyze. 

E. Error exponent bounds 

We can now proceed in a fashion that is similar to the point-to-point case and derive a series of 
exponential bounds based on Lemmas SJ [5j and [6] In the following, we present three lower bounds, the 
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random coding, the typical random coding, and the expurgated bounds. As in the case of point-to-point 
channels, here too, all the lower bounds are expressed in terms of the optimization of a single objective 
function under different constraint sets. 

Theorem 2. Fix a finite set U, P X yu G V n {X x y x U) such that X - U - Y, R x > 0, R Y > , 6 > 0, 
2 n(R x -8) < Mx < 2 nR x , 2 n ( R Y~t) < M Y < 2 nRY , and u e T Pu . Consider the ensemble, C, of multi-user 
codes consisting of all pair of codebooks (Cx,Cy), where Cx = {xi, x 2 , xm x } C Tp x|t7 (u) and Cy = 
{yi7Y2, — ,yAfy} C Tp Y]u (u). The expectation of the average probability of error over C is bounded by 

2-n[E rL (R. x M Y ,W,P X yu)+SS] < p £ < 2~ n { E r( Rx < R r ,W,Pxyu)-GS] ^g 2 ) 

whenever n > ni(\Z\, \X\, \y\, \U\,S), where 

E r (R x ,R Y ,W,P X Yu) =romp=x,Y,XY E r (R x ,Ry,W,P UX Y,V r p ), (83) 

E rL (R x ,R Y ,W,P X Yu) = minp=x,Y,XY E r p> L (R x ,R Y> W,PuxY,V r L )- (84) 

Proof: The proof is provided in the Appendix. ■ 
Corollary 5. In the low rate regime, 

E rL (R x ,R Y ,W,P XYU ) = E r {R x ,R Y ,W,P XYU ). (85) 
We call this rate region as the critical region for W. 

Proof: The proof is similar to the proof of corollary [l] and is omitted. ■ 

Theorem 3. Fix a finite set U, P XYU G V n (X x 3> X U) such that X - U - Y, R x > 0, R Y > , S > 0, 
2 n(R x -5) < Mx < 2 nRx , 2 n{RY - 5 ^ < M Y < 2 nRY , and ueT Pu . The average probability of error for almost all 
multi-user codes C = C x x Cy, C x = {xi,x 2 , ...,x A / x } C T Px|[; (u) and Cy = {yi,y2, ...,yM Y } C T Py]V {u), 
in ensemble C, satisfies the following inequalities 

2~n[E TL (Rx ,R Y ,W,Pxyu)+7S] < >q ^ < 2-"[E T (Rx ,Ry ,W,Pxyu)-GS] 

whenever n > n\{\Z\, \X\, \y\, \U\, 5), where 

E T {R X ,R Y ,W, P XYU ) 4 mm 0=XtYiXY ET{R Xl R Yl W, P UX y,Vj) (87) 
E TL (Rx,Ry,W,PxYu)=min 0= x,Y,XYEf L (Rx,RY,W,P U xY,V^ L ). (88) 

Proof: The proof is provided in the Appendix. ■ 

Corollary 6. For every finite set U, P X yu G V n {X X y X U) such that X - U -Y , R x > 0, R Y > 0, 

E T (Rx,Ry,Pxyu,W) < E^ v (R x ,Ry) < Etl(Rx,R Y ,Px Y u,W). (89) 

Proof: The proof is very similar to the proof of Corollary [2] ■ 

Corollary 7. In the low rate regime, 

Etl(Rx,Ry,Pxyu,W) = Et(R X ,Ry,Pxyu,W). (90) 

Proof: The proof is similar to the proof of Corollary [I] and is omitted. ■ 
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Theorem 4. For every finite set U, Vxyu € V n (X x y X U) such that X - U - Y , R x > 0, R Y > 0, S > 0, 
and u G Tp^, f/zere exists a multi-user code 



C {ix,.y,. />„.:/ 1 \r x .j 1 My) (91) 

G 7p v 

MAC W : X xy -> Z 



itfii/z x l G Tp x|Lr (u), G Tp y|!7 (u) /or aZZ i and j, M£ > 2 " (H * s) / and M Y > 2 " (a J S) , such that for every 



e(C,W) < 2~ n ^ E<!i:< - Rx - Ry -W-Pxyu)-^] ^2) 
whenever n > ni(\Z\, \X\, \y\, \U\,S), where 

E ex {R x ,R Y ,W, Pxyu) = min^ =x,y,xy E^ x (R x ,Ry, W, PuxY:Vp x )- (93) 

Proof: The proof is provided in the Appendix. ■ 
This exponential error bound can be universally obtained for all MAC's with given input and output 

alphabets, since the choice of the codewords does not depend on the channel. 

In the following, we show that the bound in Theorem [2] is at least as good as the best known random 

coding bound, found in [20]. For this purpose, let us use the minimum equivocation decoding rule. 

Definition 11. Given u, for a multiuser code 

C = \i.x,.y, .l),, s : i 1. ...M x ,j = 1, ...My} 
we say that the are minimum equivocation decoding sets for u, if z G implies 

HiXiYjlzu) = minfl'(x fe y / |zu). 
It can be easily observed that these sets are equivalent to a-decoding sets, where a(u,x, y, z) is defined as 

a(V UX Yz) = H V (XY\ZU). (94) 
Here, Vuxyz is the joint empirical distribution of (u, x, y,z). 

Theorem 5. For every finite set U, Vxyu G V(X x y xU) , Rx > 0, Ry > 0, and W : X x y — > Z, and an 
appropriate a-decoder (minimum equivocation), 

E r p{Rx, Ry, W, Pxyu) > E^ U (R X ,R Y , W, Pxyu) P = X,Y, XY, (95a) 

E T p {R x ,R Y , W, Pxyu) > E^ V (R X ,R Y , W, P X yu) P = X,Y, XY, (95b) 

E?(R x ,Ry,W, Pxyu) > E^ U {R X ,R Y , W, Pxyu) P = X, Y, XY. (95c) 



Hence 



E r (R x ,R Y , W, Pxyu) > E^ U (R X ,R Y , W, P X yu), (96a) 
E T {R X ,R Y , W, Pxyu) > E? U {R X ,R Y , W, Pxyu), (96b) 
E ex (R x ,R Y ,W, Pxyu) > E^ U (R X ,R Y , W, Pxyu), (96c) 

for all Pxyu £ V{X x y x U) satisfying X - U -Y. Here, E^ m is the random coding exponent of |20K . E^ u 
are also defined in [20/ for f3 = X,Y, XY. 
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Proof: The proof is provided in the Appendix. ■ 
We expect our typical random coding and expurgated bound to be strictly better than the one in 112011 
at low rates. This is so, because all inequalities in J70al l- I|70c|l and l|72al l- (|72cb will be active at zero rates, 
and thus (due to continuity) at sufficiently low rates. Although we have not been able to prove this fact 
rigorously, in the next section, we show that this is true by numerically evaluating the expurgated bound 
for different rate pairs. 

V. Numerical result 

In this section, we calculate the exponent derived in Theorem [4] for a multiple-access channel very 
similar to the one used in |20|. This example shows that strict inequality can hold in < 195cl l. Consider a 
discrete memoryless MAC with X = y = Z = {0,1} and the transition probability given in the following 
table. 



X 


y 


z 


W(z\xy) 











0.99 








1 


0.01 





i 





0.01 





i 


1 


0.99 


1 








0.01 


1 





1 


0.99 


1 


1 





0.50 


1 


1 


1 


0.50 



First, we choose some time-sharing alphabet U of size \U\ =4. Then some channel input distribution 
PuPx\uPy\u is chosen randomly. The following table gives numerical values of the random coding 
exponent of 11201 , and the expurgated exponent we have obtained for selected rate pairs. 



Rx 


Ry 


E ex (R x ,R Y ,W,P UX Y) 


E^(R x ,R y ,W,P UX y) 


0.01 


0.01 


0.2672 


0.2330 


0.01 


0.02 


0.2671 


0.2330 


0.01 


0.03 


0.2671 


0.2330 


0.02 


0.01 


0.2458 


0.2230 


0.02 


0.02 


0.2379 


0.2230 


0.02 


0.05 


0.2379 


0.2230 


0.03 


0.01 


0.2279 


0.2130 


0.03 


0.03 


0.2183 


0.2130 


0.04 


0.01 


0.2123 


0.2030 


0.04 


0.04 


0.2040 


0.2030 


0.05 


0.05 


0.1930 


0.1930 


0.06 


0.01 


0.1856 


0.1830 


0.06 


0.06 


0.1830 


0.1830 


0.07 


0.01 


0.1740 


0.1730 


0.07 


0.07 


0.1730 


0.1730 
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As we see in the table, in the low rate regime, we have strictly better results in comparison with the 
results of [20 [. For larger rate pairs, the inequalities containing min{i?x, Ry} will not be active anymore, 
thus, we will end up with result similar to |20|. 

VI. Conclusions 

We studied a unified framework to obtain all known lower bounds (random coding, typical random 
coding and expurgated bound) on the reliability function of a point-to-point discrete memoryless channel. 
We showed that the typical random coding bound is the typical performance of the constant composition 
code ensemble. By using a similar idea with a two-user discrete memoryless multiple-access channel, we 
derived three lower bounds on the reliability function. The first one (random coding) is identical to the 
best known lower bound on the reliability function of DM-MAC. We also showed that the random coding 
bound is the average performance of the constant composition code ensemble. The second bound (typical 
random coding) is the typical performance of the constant composition code ensemble. To derive the third 
bound (expurgated), we eliminated some of the codewords from the codebook with a larger rate. This 
is the first bound of its type that explicitly uses the method of expurgation in a multi-user transmission 
system. We showed that the exponent of the typical random coding and expurgated bounds are greater 
than or equal to the exponent of the known random coding bounds for all rate pairs. By numerical 
evaluation of the random coding and the expurgated bounds for a simple symmetric MAC, we showed 
that, at low rates, the expurgated bound is strictly larger. All these bounds can be universally obtained 
for all discrete memoryless MACs with given input and output alphabets. 

Appendix 

1. Point to Point Proofs 

This section contains the proof of all lemmas and theorems related to point to point result. 
Proof: (Lemma Q) We use the method of random selection. Define M such that 

2 n(R-S) < M < 2 nR . 

In the following, we obtain the expectation of the packing function over the constant composition code 
ensemble. The expectation of tt(X m ,V xx ) can be obtained as follows: 

1 M 1 M 

E (n(X M , V xx )) = -^^E(l v (XuXjj) = - EE P (X S E Ty^jX,)) 

= (M - 1)P (x 2 £ T VjE|x (Xij) < 2«(«- r v(XAX)+<5) > (97) 
Similarly, it can be shown that for sufficiently large n, 

E (ir{X M , V xx )) > 2 «(«-M*AX)-<5)_ (9g) 

The expectation of A over the ensemble can be written as 

M 

E(X(X M ,V xxjt )) = ^EE E V((Xi,Xj,Xk) e T VxXX ) . (99) 
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Since 

2n[H(XX\X)-5] 2 nH (XX\ x ) 

= =r- <F((Xi,Xj,X k ) GT V _J < = = , (100) 

2nH(X)2nH(X) ~ U ' J ' ; v xxxJ - 2n[H(X)-8]2n[H(X)-S] ' V ; 

it can be concluded that 

2 ^s(R,v xxx )-2S] < E ( X ( X ai \v xxx )) < 2"' Es W^«)+^ i (101) 

where 

E S (R, V xxx ) ~ 2R ~ !(X AX) — I(X A XX). (102) 
By using &7\ and markov inequality, it can be concluded that 

P (tt(X m , V xx ) > 2 »<*-M*a*)+3*) f y \ < y K(n(X M ,V xx )) < 2 _ nf 

V V ' XXI - XX J - ^ r>n(R-I v (XAX)+26) - ' V 7 

therefore, there exists at least one code, C r , with M codewords satisfying 

<° r i v xx) < 2 n <- R - Iv <- XAX ^+ 25 \ (104) 

■ 

Proof: (Lemma |2) To prove that a specific property holds for almost all codes, with certain number 
of codewords, in the constant composition code ensemble, we use a second-order argument method. We 
already have obtained upper and lower bounds on the expectation of the desired function over the entire 
ensemble. In the following, we derive an upper bound on the variance of the packing function. Finally, 
by using the Chebychev's inequality, we prove that the desired property holds for almost all codes in 
the ensemble. 

To find the variance of the packing function, let us define Uij = 1t v _ {Xi,Xj), and = Uij + Uj\. 
We can rewrite ir{X M , V xx ) as 

M M M 

i— 1 j^i i—1 j<i i—1 j<i 

It is easy to check that Yy's are identically distributed pairwise independent random variables. Therefore, 
the variance of tt(X m , V xx ) can be written as 

i M i f]\{\ 

Var(n(X M ,V xx )) = ^EE^(^) = WA 2 ) Var ^- ^ 

i=l j<i ^ ' 

To find the variance of Y21, let us consider the following two cases for V xx . 
• Vxx * s a symmetric distribution. In this case U12 = U21, therefore, 



Y 2 i 



2 with probability p < 2-™[ 7 ( XA *)- 5 l 
with probability 1 — p 

and the variance is upper bounded by 

Var{Y 21 ) < E(Y 2 \) = 4 x 2-™[ / (* A *)- 5 l (107) 
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• ^xx * s not a symmetric distribution. In this case, if JZy = 1 =£• Uji = 0. Therefore, 

P (Y12 = 1) = P (U12 = 1 or U 2 x = 1) = P (Z7i2 = 1) + P (U21 = 1) < 2 x 2-"[ 7 ( XA *)- <5 l , (108) 
therefore, 

Far(F 2 i) < = 2 x 2-™[ / ( XA *>-' 5 ]. (109) 

By using (107L and jl09i , we have 

Var (n(X M ,V xx )) < ^(f) 4 x 2-^^ XAX ^ < 2 x 2 ~ n ^ XA ^- s \ (110) 
for any V^-^ £ "Pf^ x A"). Now, by using Chebychev's inequality, 

P (\tt(X m , V xx ) - E (tt(X m , V^)) I > 2" 5 for some 
< J] P {\tt(X m , V xx ) - E V xx )) I > 2" 5 ) 

^xx 

Var (ir{X M , V xx j) ^2x 2-"[ J (* A *)-*] 



xxl 



< 



E : ui v"v vv ' y xxJj < \p 
02nt5 — 



22ni5 — / j <£ln& 



= Yj 2x 2- n{I{x " x ^ < 2~ n i, for sufficiently large n. (Ill) 

Moreover, by using (TUT) and Markov's inequality, it can be concluded that 

\(X M V - - )> 2^ E ^ R ' v xxx)+^} for some V - - \ < V EX ^ xM ' V xxx) < 2 -n5 Q12) 
v ,v X xx)^ z ror some ^ ^ on[_E s (i?,v XJf ^)+45] - z ' 

v xxx 

Now, by combining jllli and (|112l l and using the bound on E (tt(X m , V xx )), we conclude that for any 
V xx e V{X x X), any V^xje G V(X x X x X), for sufficiently large n 

2n(R-I(XAX)-5) < ~(X M y _ \ < 2n(R-i(A*AX)+<5) 

V ' XX ' ' 

A(X M ,V^) < 2"[^(«'^xx)+45] 7 (113) 
with probability > l-2x2 _n 5. We put all the codebooks satisfying 1 11131 in a set called C T . 

■ 

Proof: (Lemma© Consider the code C r = {xi,X2, ...,xm} whose existence is asserted in random 
coding packing lemma. Let us define 

n(C) 4 £ 2 -n{R-Iv{XAX)+Z8)^ C r^ y^y (n4) 
^xx 

Note that using LemmaUJand using the fact that n(C r ) = ^ Ei^i {Ev xjf |Ty A |x (x<) n C-| 2 -n(fl-M*AX)+3<5) ^ 
it can be concluded that 

n(C r ) < 2~™ (fl ~ /v(XA * )+3l5) 2 n( - R ~ /l ' (XA ^ )+2 ' 5) < - (115) 
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As a result, it can be concluded that there exists M* > 4f codewords in C r satisfying 

^ \T v ^ x (*i) n C r |2-"( R -^(^x)+35) < L (116) 

^xx 

Let us call this subset of the code as C ex . Without loss of generality, we assume C ex contains the first 
M* sequences of C r , i.e., C ex — {xi, x 2 , Xm« }• Since 

\Tvx ]x (xi) nc ex \< \T Vx]x (^) n C r | Vx, G C e *, (117) 

it can be concluded that for all Xj G C e:c , 

53 |7V^ |Jt (xi) n c^-^K-M-XAiHM) < L (118) 

^xx 

Since all the terms in the summation are non-negative terms, we conclude that 

\T V (xi) r\C ex \ < 2 n ( R - Iv ( XA ^+ 3S \ (119) 
for all V X x e v ( x x x )> and a11 x * e ° ex - Also ' b Y ES), it can be concluded that for all V xx G P(A? x X) 

1 M * 

tt(C-,^) = — 53|T Vje|X (x0nC ea: |<2"( fl -^( XAX )+ 35 ). (120) 
i=i 

■ 

Proof: (Fact|l) We will use the result of Lemma [j] and the relation between the probability of error and 
the packing functions. Let X M = (Xi,X2,...,Xm) be independent sequences of independent random 



variable, where XiS are uniformly distributed on Tp. 

(Upper Bound): Taking expectation on both sides of d23l , using Lemma [1] and using the continuity of 
information measures, it can be concluded that 

E(e(X M ,W))< 5] 2-"[ D ( vv ^ll M/ l p ) + l / ( XAXY )-- R l + -' 5 ] 
v XXY eK 

< 2 -n[E r (R,P.W)-28] Q2\) 

whenever n > ni(\X\, \y\,S), where 

EJR,P,W)= min D(V Y \ X \\W\P) + \I V (XY AX) - R\ + , (122) 



and V r is defined in 

(Lower Bound): Taking expectation on both sides of |26j, and using Lemma [1] we have 
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P e = Ee(X M ,W) > 2 ~" 



[D(Vy tx \\W\P)+I v {XAY\X)+8] 



2n(R-I(XAX)-S) 



2-n[Iv (XAY\XX)]2n(2R-I(X/\X)-I(X/\XX)+AS) 



XXY~ y XXY 



v XXXY 

v Y * v =v. 



y 2- n ^ D( ' V Y\x\\W\P)+I v (XAXY)-R+2S} 



-n[7 v (JfAJfXY)-fl+35] 



(123) 
(124) 



Toward further simplification of this expression, we have the following lemma. 
Lemma 7. 

min I(XAXXY) = I[X AXY). (125) 

^XXXY : 
VxXY—^XXY 

Proof: Note that, for any V XXXY , 

I(X A XXY) = I(X A XY) + I(X A X\XY) > I(X A XY), (126) 

therefore, 

min I(XAXXY) >I(XAXY) = I(XAXY). (127) 

^XXXY : 
^XXY—^XXY 

Now, consider V* - - defined as 

X J\ J\ Y 

V XXXY^ X ^^^^ = v x\XY( i \ x -.y) v x\XY^\ x -'V) v XY{x,y). (128) 
Note that V*~ v = V*^ v , and X - (X,Y) - X. Therefore, 

I V *{X A XXY) =I V (X A XY) = I V (X A XY). (129) 
By combining j!27i and j!29i , the proof is complete. ■ 



Therefore, using the above lemma, fl!24Jl can be rewritten as 

p e > ^ 2- n[D( - VY i xllw \ p)+Iv ^ AXY) - R+3S] . (130) 

I(XAXY)>R+3S 

By using the continuity of information measures, it can be concluded that 

E (e(X M ,W)) > 2-^ E ^ R > p W+ u \ for sufficient large n (131) 

where 

E L (R,P,W)= min D(V Y \x\\W\P) + 1 V [XY A X) - R. (132) 

VxXY^ 

I(XAXY)>R 
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Now, by using Markov inequality and Q2l) , we conclude that 

P (e{X",W) > < 2 E jg^ff 3g] < 2- s . (133) 

Therefore, with probability greater than 1 — 2~ nS , any selected code with M codewords form the constant 
composition code ensemble satisfies the desired property Let us call one of these codebooks as C r . ■ 
Proof: (Corollary [l) Consider the input distribution P* £ V{X) maximizing the random coding 
bound, i.e., 

P*=arg max E r (R,P,W). (134) 
Pev(x) 

Let us define 

V xxy = ar § min E r( R , P*,W). (135) 

*X X Y 

For any R < R cr u, the random coding bound is a straight line with slope —1, and the term in | • |+ is 
active. Therefore, 

E r (R, P*,W) = D(VJ [X \\W\P*) + I v * (X A XY) - R. (136) 

Here, ly (X A XY) > R. It is clear that V*g y is the minimizing distribution in E rL (R, P*,W), and as a 
result 

E rL {R,P*,W) = E r {R,P*,W). (137) 



Proof: (Theorem|l) In the proof of Fact[H we used the lower and upper bounds on the expected value 
of he first-order packing functions and an upper bound on the expected value of the second-order packing 
functions. In the following, we use similar techniques on the packing function of almost every codebook 
in the ensemble by using the bounds obtained in Lemma Consider the code C whose existence is 
asserted in the typical random coding packing lemma. For all V X x 6 V{X x X), we have 

1 M 

^^|T^ |x ( Xl )nC| < 2»(«-M*ax) + 2^ (13g) 

i—l 

By multiplying both sides of inequality dl38l > by M , and using the proper upper bound on the number 
of sequences in C, we conclude that 

\T v ~ lx (x t ) nC| < 2 ni2R - Iv ^ x ^ +2S ^ V* = l,...,Af, (139) 

for all V xx £ P(X x X). We will obtain a higher error exponent for almost all codes by removing certain 
types from the constraint set V r n . Consider any V xx £ V(X x X) satisfying I V (XAX) > 2(R+6). By flgg l, 

\T v . ]x (xi)nC\ =0foraUz w{C, V xx ) = 0. (140) 

Upper bound: Hence, by using (p3l on C, and by using the result of Lemma |3 we have 

e(C, W) < ^2 2~ n[D{VY * x ^ wlP)+lIv( - XYAjt) ~ R \ + - 25] 
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where 

(*) = {Vxxy e x A" x ^) : V* = V* = P, 7y(X A X) < 2R + 26 , a{P, V Y[St ) < a(P, V Y \ X )}(U1) 

Using the continuity of information measures, the upper bound as given by the theorem follows. 
Lower bound: Using j26t on C and using Lemma [2J we have 



e(C,W)> 2- n ^ D{ - VY \ x ^ w \ p ^ +Iv{kfxY \ x ^ 



" XXX' 



■n[D(y Y ix\\W\P)+Iv(XAY\X)+6] 



^XXXY : 
*XXY = VxXY 

2n(R-I(XAX)-S)_ 

2-n[I v (XAY\XX)}2n(2R~I(Xf\X)~I(Xr\XX)+2&) 



VxXXY- 
^ XXY = ^XXY 



y 2- n { D ( V Y\x\\W\P)+I v (XAXY)-R+28\ 



i - J2 2 ~ n[I 



v(XAXXY)-R-3S] 



> 2- n [ D< - y r\x\\ w \ p ')+ I v( x ^ XY )-R+3S] 



(142) 
(143) 
(144) 



VxXY^nW 

I(XAXY)>R+5S 



Here, the last inequality follows from Lemma [7] 

By using the continuity argument, and for sufficient large n, 



i(C,W) > 2- n ^ ELT{ - R ' P ' W ^ +AS \ 



where 



E LT {R,P,W) = min D(V Y \x\ I W\P) + I v (XY A X) - R. 



(145) 
(146) 



y XXY* 

I(XAXY)>R 



Proof: (Corollary |2]l Fix R > 0, S > 0. By the result of Theorem [TJ and for sufficiently large n, there 
exists a collection of codes, C* , with length n and rate R, such that 
. P(C) > 1-5, 

. 2 -n[B Ti (i?.,P,M/)+45] < e(C; jp) < 2 -n[^r(H,P,W)-3«] f or a H £ £ C*. 

Note that 

max min — loge(C, W) > min — loge(<7, W) > E T (R, P, W) - 36. (147) 

C:P(C)>i-<5 cec n Cec* n 

Now, consider any high probability collection of codes with length n and rate R. Let us call this collection 
as C. Note that 



P(C*) >l-6 

p(d) > i - <j 



»(C*nC) > i-2<5^c*nc^ 



(148) 
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Consider a code C(C) e C* n C. It can be concluded that 

max min-iloge(C,WO < max -- log e(C(C), W) < E LT (R, P, W) + 45. (149) 
c-.p(c)>i-s cec n c-.f(c)>i-s n 

The last inequality follows from the fact that C(C) € C* . By combining | |147t and (1491 1, and by letting 5 
goes to zero and n goes to infinity it can be concluded that 

E T (R, P, W) < El v {R) < E TL (R, P, W). (150) 

■ 

Proof: (Fact [2) First, we prove the following lemma. 

Lemma 8. Let C ex be the collection of the codewords whose existence is asserted in Lemma\3\ For any distribution 
v xx £ "Pn(X x X), satisfying ly [X A X) > R + 5, the following holds: 

ir(C ex : V X x)=0. (151) 

Proof: By (32), 

\T V]t (in) r\C ex \ < 2 n( - R - Iv ^ XA ^+ 2S \ (152) 
for every x> € C ea: . Since /^(X A X) > R + 25, it can be concluded that 

\T V - ]x (xi) n C CT | = for every x 4 G C ex Tr(C ex , V xjt ) = (153) 

■ 

The rest of the proof is identical to the proof of random coding bound. ■ 
2. MAC Proofs 

Proof: (Lemma|D In this proof, we use a similar random coding argument that Pokorny and Wallmeier 
used in fl9l . The main difference is that our lemma uses a different code ensemble which results in a 
tighter bound. Instead of choosing our sequences from Tp x and Tp Y , we choose our random sequences 
uniformly from Tp x|[7 (u), and Tp y , t7 (u) for a given u 6 Tp u . In |20| , we see a similar random code 
ensemble, however, their packing lemma incorporates the channel output z into the packing inequalities. 
One can easily show that, by using this packing lemma and considering the minimum equivocation 
decoding rule, we would end up with the random coding bound derived in [20|. 

Fix any U, P X yu £ V n {U x X x y) such that X - U - Y, R x > 0, R Y > , S > 0, and u e T Pu . Define 
M x , M Y such that 

First, we find upper bounds on the expectations of packing functions for a fixed a and V UXYXY , with 
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respect to the random variables Xj and Yj. Since XiS and YjS are i.i.d random sequences, we have 

1 



E 



N„{X M * x Y My ,V UX y) 



= E 



M X M Y 



^t Vuxy {^X u Y 3 ) 



EflT^KX^Fx)] 

£ lTv xy|u (x, y|u)P(X x = x|u)P(r x = y|u) 



2-n[H v (X|C/)-<5]2-n[H"v(i'|U")-5] 

(x,y)GT Vxy|u(u ) 
< 2nH v (XY\U)2-n[H v (X\U)-S]2-n[H v (Y\U)-5] 

_ 2-n[Iv(XAY\U)-26] _ 2 -n[ F u{VuxY)-26] 



On the other hand, 



E 



Nu{X M * xY M \V UXY )] =^lrv xy|a (x,y|u)P(X 1 =x|u)P(y 1 =y|u) 



XY\U 

2- nHv( . x \ u '>2~ nHv( - Y \ u ^ 

(x,y)eT Vxi , |Lr (u) 
> 2 n [Hv(XY\U)-8]2-nH v (X\U)2-nH v (Y\U) 

_ 2-n[Iv(XAY\U)+6] _ 2- n [ F u(VuxY)+S] 



Therefore, by J 1541 and \155\ , 

2-n[Fu(VuxY)+$] < g 

By using a similar argument, 



< 2-n[Fu(Vux Y )-2S] 



(154) 



(155) 



(156) 



E 



> 2 -n[F x (y uxYX )-AS]_ 



(157) 
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On the other hand, 



E 



N X (X M * x Y M y,V UXYjt )] > (M X - 1)e[i Tvuxy (u,X 1 ,Y 1 )1 TVuxy _(u,X 1 ,Y u X 2 ) 

= (M X - l)J2nXi = x|u)P(Yi = y|u)l Tvaxv (u,x,y) 

• VP(I 2 =x|u)1 3V fu,x,y,x) 

X 

> (Mjc - 1) ^ 2 -™^( x l' 7 )2~"- ffv(y|,7) 

x,yery xv|E/ (u) 

2 -nH v (X\U) 



xeTv- (u.x.y) 



> (M x - i)2 n[H(XY \ U) ^ 5] 2^ nHv{x ^ J) 2^ nHv{Y],u) 

. 2 n[fMX|£/ Xr)-5]2-nHv(X|i7) 
-> 2-n[-fv(^AY|t/)+Z v (XAy|!7)+7v(XAX|i7y)-iIx+35] 

XYX 

)+3<5] 



Therefore, by | [T57t and ( |T58l l, 

2 -™[F x (^ xrx )+ 3 ' 5 ] < 



< 2 -n[F x (y Lrxl , x )-45]^ 



(158) 



(159) 



By using a similar argument for N Y (X Mx x F Y ,V UXYX ) and N X y(X x x ^ > Wxy)' we can 



show that 



)+3<5] < ] 

2 -n[Fxy(V r !7xrx? )+45] < jg 



< 2- n Ifti F ( v uxri?)- 4i ], (161) 
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We can obtain an upper bound for E [A XY (X Mx x Y My , V UXYXYXY )\ as follows 
E[AxY(X M *,Y M -,V UXYXYxt )} 



E 



M x M 



T7~EE E 1t - -wfaXuY^XtoYuXvM) 

■My ^ ^ UXYXYXY J 



W l'jtj,l 



< M X M Y E \l T VuxY (u, X x , Y{)l Tv (u,X 1 ,Y 1 ,X 2 ,Y 2 ,X 3 ,Y 3 

L V UXY UXYXYXY 

= M\M Y P (^i = x ' y i = y» X ? = x, Y 2 = y, X 3 = x, Y 3 = y|u) 

x,y,x,y,x,y 

• 1t v UX y ( U ' X ' y)- lT ^x^x?xl> (U ' X ' y ' *' y ' *' A) 

= M X M Y 2 F(X 1 = x|u)Pr(H - y|u) • l Tvuxy (u, x, y) 
x,y 

• ]T P(*2 = x|u)lr Wj _ (u, x, y, x) £ P(F 2 = y|u)lT Wjf? (u, x, y, x, y) 

x y 

• VP(I 3 = x|u)l Tv . (u,x,y,x,y,x) Vp(7 3 = y|u)l Tv (u, x, y, x, y, x, y) 

"(, IrIrx V UXYXYXY 

x y 

< M X M Y ^2 2~ n[Hv< - x \ u) ~ 5 h~ n[Hv( - Y],u) ~ s] ^2 2~ n[Hv( x\ u) ~ s] 

x,yerv XY|a (u) ieT % |ray (u,x,y) 
YCzTV- , - (u.x.y.x) x£E7V- ~ - (u.x.y.x.y) 

J Y\UXYX y ' ,J 1 X\UXYXY X ' J J 1 



(u,x,y,x,y,x)2 



-n[H v (Y\U)-S} 



y^^YlUXYXYX 



_ 2- r A H v{y\U)~S]2nH v (X\U XY XY)2-n[H v (X\U)-8]2nH v (Y\UXY XY X)2-n[H v (Y\U)-8] 
< 2-n[I(XYAXY\U)+I(XYAXYXY\U)+I(XAY\U)+I(XAY\U)+I(XAY\U)-2Rx~2RY~68] 

_ 2-™[ E s Y ( v uxyxyxy)- 6S ] , (162) 

By using a similar argument, we can obtain the following bounds 

E [A x (X Mx x Y My ,V uxyxx )] < 2-^ E s( v uxYxx)-is] (163) 
E [A Y (X Mx x Y My ,V uxyyy )] < 2- t ' [e ^ ( - v uxyyy)-^] (164) 

Here, Eg, E Y and E§ Y are defined in ((67aT >- ((67ct . 
By using Markov inequality, it can be concluded that 

P (Nu(X Mx x Y My ,Vuxy) > 2- n[Fu(VuxY) - 3S] for some V UX y) 

x , E(N„(X Mx xY My ,V UXY )) k - a « 

^ E 1 2-n^(W)-3fl ^ ^ E 2^<2-« 5 (165) 

VirXY- VUXY- 

Vux—Pux Vux—Pux 

VuY—PuY VuY—PuY 
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Similarly, it can be shown that 

P (n x (X Mx x Y My ,V uxyx ) > 2- n ^ Fx< ^uxYx)-^] f or some V UXY x) < 2 ~™^ (166) 
P (n y (X Mx x Y My , V uxyy ) > 2 -«[*VOWy)-5.5] for some V UXYY ) < 2~ n i , (167) 
P [n X y{X Mx x Y My ,V uxyxy ) > 2-"l f "(Wy)-5»'] for SQme V UXYXY ) < 2~ n *. (168) 
Now, by combining | |165H168> , and using the union bound, it can be concluded that 

v(Nu(X Mx x Y My ,V UX y) > 2- n[FuiVuxY) ^ &] for some V UXY or 
iV x (X M * x Y My ,V uxyx ) > 2- n ^ Fxi - v uxYx)-^] for S o m e V UXYX or 
iV Y (X Mx x y Mr ,V^ x ^) > 2- n[FY{v vxvY)-5S] for SQme v^^yy. or 

iVjfy(X Mx x r^.^y) > 2-^ Fxy ^ v uxyxy)-5S] for gome ^^.j < 4 x 2 -»4, (169) 
therefore, there exists at least a multi-user code with the desired properties mentioned in W(& - $77\ . 

■ 

Proq/: (Lemma [5) To prove that a specific property holds for almost all codes, with certain number 
of codewords, in the constant composition code ensemble, we use a second order argument method. We 
already have obtained upper and lower bounds on the expectation of the desired function over the entire 
ensemble. In the following, we derive an upper bound on the variance of the packing function. Finally, 
by using the Chebychev's inequality, we prove that the desired property holds for almost all codes in 
the ensemble. To find the variance of N v {X Mx x Y My ,V UX y), let us define W lj = ^t Vuxy (u, Xi,Yj). 
Therefore, the variance of Nu(X Mx x Y My ,Vuxy) can be written as 



Var (Nu(X Mx x Y My , V UXY )) = Var I — 1— ]T It Vuxy (u, X it Y 3 ) 



(170) 




Since Wy 's are pairwise independent random variables, (|170b can be written as 

Var (N v (X Mx x Y My , V UX y)) = jj-r^ £ Var (W tj ) 



< 



Ml Ml 

X 1,3 



< 1 2- n ^ Fu{ yuxY)-2S] < 2- n l F u{Vu xy)+Rx+Ry -25} 

~ M X My ~ ■ \ ) 

By defining Q\ k = l Tv . (u, X t , Y 3 ■, X k ), the variance of N x (X Mx x Y My ,V UXYX ) can be upper- 
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bounded as follows 



Var{N x (X»* x Y M ^ Wjf )) =Var l J ^Y,J2 1 T VuxY ^^,Y J ,X k )\ 

K Y \ i,j k^i ) 

K Y \ 3 i kj&i J 

»m v >* (SEE % + «.) - jirsfV- (VVV ';.') , 



(172) 



where J/ fc = Q^. + Q^, k < i. One can show that Jf fe 's are identically pairwise independent random 
variables. Therefore, the Far (iV x (X M * x F Mi -, V UXYX )) can be written as 

x Y j i k<i Y 

To find the variance of J\ u let us consider the following two cases for V UXYX : 

* Vuxyx i s a symmetric distribution, i.e., V UXYX = V UXYX . In this case Q\ 2 = Q\ x , therefore, 



J 2,l — 



2 with probability p « 2-™[ / ^ XAy l c/ )+ / ^ AXy l c/ )] 
with probability 1 — p 

and the variance is upper bounded by 

VaHJl,) < EiJl, 2 ) = 4 x 2 -»[M*AV|iO+M*A;nr|y)] j (174) 

• ^[/xyi ^ s n °t a symmetric distribution. In this case, if <5^ fc = 1 = 0. Therefore, 

P (4i = 1) = P (Ola = 1 or Qli = 1) = F (Q12 = 1) + P (Qai = !) 

< 2 x 2"™[ /l '( XAy l t/ )+ /l '(^ AXr l c/ )] (175) 

therefore, 

VaKJ^) < ^(J^ 2 ) - 2 x 2-4^(XAr|Cf)+iv(XAxr| C /)]_ (176) 
By combining the results in dl73H175l l, it can be concluded that 

Var (N X (X M - x Y M * ,V UXYX ) < 2 -n[M*Ay|E/)+iv(XAxr|E/)+*Y-35]. (177) 

Similarly, it can be shown that 

Var (N Y (X Mx x Y My , V UXYY )) < 2 -™[^(XAr|C7)+iv(?Arx|a)+fl x -35]_ (17g) 

By defining = l Tv _ (u, X l7 Y h X k ,Yi), the variance of N XY (X Mx x F Mi - , V^ xr ^y-) can be upper- 



37 



bounded as follows 



/ 



Var (N XY (X Mx x Y My ,V UXYXY )) = Var 



MxM: 



v 



i,j k=£i 



M X M Y 



Xu,Xi,Y jt X k ,Yi) 



UXYXY 



EE 1 

l,j k=£i 

\ i j k<i tyj J 



M x M y 



X Y \ i 3 k<i j<l I 



(179) 



where Sf k = R 3 ik + R J ki + R? k + R k \, k < i, I < j. It is easy to check that Sf' k 's are identically pairwise 
independent random variables. Therefore, the Var (Nxy(X x X Y My , V UXYXY )) can be written as 

Var (N XY (X M * x Y My , V UXYXY )) = EEEE^ ^ ■ ( 18 °) 

X Y i j k<i Kj 

By using a similar argument to l|174|l - l|175|l , the variance of To find the variance of S-[ 2' can ^e upper 
bounded by 



Var(s\'f} < 16 x 2-™[M^AY|io+M*Ay|a)+M*} > AXY|t/)-4<5] i 



(181) 



and therefore, 

Var (N XY (X Mx x Y My , V UXYXY )) < 4 x 2-"[^(^|t/)+/v(XAy|c/)+^(XKAxr| C /)-4 (5 ]_ (lg2) 
Now, by using the Chebychev's inequality, we can obtain the following 

P (\Nu{X Mx x Y My , Vuxy) - E (Nu(X Mx x Y My , %y)) | > 2 2nS for some Vuxy) 

^ E P {\ N u(x Mx x y My ,y rar ) - e (Nu(x Mx x r M ^, v UXY )) \ > 2 2nS ) 

Vuxy 

Var (N v {X Mx x Y My ,V UXY )) 



* E 



24ni5 



(183) 
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—n5 

(186) 



Similarly, it can be shown that 

P(\N x (X Mx x Y My ,V UXY x) -E(N x (X Mx x Y My , V uxyx ))\ > ^ for some v uxyx) < ^ ( 184 ) 
P (\N Y (X Mx x Y My , V UXYY ) - E (N Y (X Mx x y M ^ , V UXYY )) | > 2 2n5 for some V UXYY ) < 2~™ 5 (185) 

P (\N XY (X Mx x , V UXYXY ) - E (iV X y (X Mx x y M - , V^^)) | > 2 2nS for some V UXYXY ) < 2 

flf 

Now, by using the result of Lemma [4] and Markov's inequality, it can be concluded that 

p(a x (x Mx x m -,v uxyxx ) > 2-»K(w*x)-5*) for some Vuxy .^ 

< v(k x (X Mx x MY ,V UXYXX )>2-< E s( v ux Y Y*)-^ 

VjJXYXX 

Similarly, 

P (A y (X Mx x F My , V^yyy) > 2-™K (>^xyy*)-5«) for some Vfr) ^ 2 ~™^ ( 188 ) 

and 

P (kxv{X Mx x V^y^y) > 2-< E S Y ( v uxvxfxt)^) for some v^y^y) < 2""i (189) 

Therefore, with probability > 1 — 7 x 2~™5, a code C = Cx x CV from random code ensemble satisfies 
the conditions given in the lemma. ■ 
Proof: (Lemma [6) Let C r x = {xi, x 2 , x« x } and Cy = {yi,y2, ...jM y } be the collections of 
codewords whose existence is asserted in Lemma [4] Let us define 



U(C r x xC r Y )^ ]T {n v (C x xC y ,V UX y)2 



n[Fu{Vux Y )-6S] 



+ N X {C X x C Y ,V UXYX )2 n ^ Fx( - v uxYx)-es] 
+ N Y (C r x x C y ,V uxyy )2^ Fy ^xyy)-^] 

+ N X y(C x x Cy, V c/X y^y)2"^^(^^**-)- 65 ]} (190) 

< ^ 4 x 2- n<5 < - (191) 

V{j XYXY 

For C r = C x x Cy, and the sequence u defined in random coding packing lemma, we define 

LuiCWuxY^ij) = l TvuxY (vi,Xi,yj), (192) 

L x (C r ,V UXYX ,i,j) = Yl, lT v UXYjt Kxi.yj.x*). ( 193 ) 

M^Wy.U) = E^W^u^^y,,^), (194) 

W^^^y^y.^i) = 5Z5^l5v pjryjey (u,3t i ,y i ,x fc ,yi). (195) 
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By definition of N a , dl90l l can be written as 

M x 

Mx 



, M x 

n(C r ) = jr Y, G W< for a = U,X, Y, XY, (196) 



where G(i) is defined as follows: 

M Y 



G «-^E E {^(c r ,w,<,j)2 n[JiV(Vbxy) - M 



+L x (C r ,V UXY y,i,j)2 n ^ V uxrr)-M] 



UXYX' 
VXYYi 

+L XY (C r ,V UXYX y, i ,j)2 n ^^ V UX Y x9)-^]y {m) 

By using <|191b , we see that the average of G(i) over C r x is upper bounded by |, therefore, there must 

X' 



2 

exist Afx > codewords, e C r x , for which 



< 1. (198) 

Let us call this set of codewords as C x . Without loss of generality, we assume C x contains the first Mx 
sequences of C x , i.e., Cx = {xi, X2, x^ }. Consider the multiuser code Cf x = Cx x Cy. By definition 

of L a , a = U,X, Y, XY, 

L a (C{ x , V, i,j) < L a (C r , V, i,j) V (x„ yj ) e CT. (199) 
By combining (|198l l and (|199l l, we conclude that for all i 6 {1, 2, M x } 



J— 1 ^[/xyxy 

+^(cr.Wf. i .j) 2n[Fr(w) " 6fl 

+Lxy(C 1 ea: ,y £/xy ^ ) i,i)2"[ Fx ^(W xy)-«l) < i, (200) 



which results in 



M Y 

]r {Lu{cr,v UX Y^m n[Fv{VvxY) - RY - 6S] 

j — l VuxYXY 

+L X {CT , V UXYX ,tJ)2^ F ^ v ux Y x)-R Y -es] 
+L x {CZ x ,V UXYY ,i,j)2 n ^ V uxr^- R r-W 

+L XY {Cf\V UXYXY ^ ] )2 n ^(v U xYXY)-Rv-^ < L (2Q1) 

Since all terms in the summation are non-negative, we conclude that 

L a (CT ,V,i,j)2- n ^ v ^- RY - 65 ^ < 1 (202) 
for all i e {1,2,... ,M X }, j e {1, 2, M Y }, all V € x <Y x y x A" x y), and all a = U,X,Y,XY. 
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Therefore, 

L a (Cf,V,i,j) < 2- n ^ v ^- RY - 6S l (203) 
On the other hand, (jl91|> can also be written as 

My 

n(c r ) = jr Y. H d)> for « = c/, x, y, xy, (204) 



where H{j) is defined as 

1 Afx 

^')-^E E {M^w,u)2"^ w >- 65 ] 



- 1 - ^yxyxy 



+Lx(C r ! V [/x ^,i,i)2"^(W*)-6*] 
+L x (C r , V UXY9 , i, j)2"I^(W 

+Lxy(C r ,T/ t/xy ^, l ,j)2 Il[Fx5 ' (y — *^- 651 }. (205) 

By a similar argument as we did before, we can show that there exist My > codewords, y^ e C y , 
for which 

ff(j) < 1. (206) 

Let us call this set of codewords as Cy • Without loss of generality, we assume Cy contains the first My 
sequences of Cy, i-e., Cy — {yi, Y2, Ym y }• Consider the multiuser code C^ x — Cx x Cf?. By definition 

of L a , a — U,X, Y, XY, we have 

L a {C?, V, < L a (C r , V, ij) V (x„ yj ) G CT. (207) 
By a similar argument as we did before, we can show that 

L a {C^,V,i,j) < 2- n l F ~<y)- R x-* s l. (208) 

for all i G {1, 2, M x }, j G {1, 2, My}, all V G P(W x A" x ^ x A x }>), and all a = U, X, Y, XY. 

By combining (|203l l and (|208l l, we conclude that, there exists a multiuser code C ex = C* x x C Y with 
MjX My messages 

M£> , M Y > , M* x xM Y > - (209) 

such that for any pair of messages (x,, y 3 ) G C ex , all V G V(U x X x y x A" x y), and all a = U, X, Y, XY, 

L a {C ex ,V,i,j) < 2-^( v V mi »{«^W~65] pit)) 

It is easy to check that 

Il(C ex ) < 2 x n(C r ) < 1, (211) 

therefore, C ex , satisfies all the constraints in Il80ab - ll80db . 

Here, by method of expurgation, we end up with a code with a similar average bound as we had for 
the original code. However, all pairs of codewords in the new code also satisfy (|81al l- (|81dt . Therefore, 
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we did not lose anything in terms of average performance, however, as we will see in Theorem |2 we 
would end up with a tighter bound since we have more constraints on any particular pair of codewords 
in our codebook pair. ■ 
Proof: (Theorem [2) Let us do random coding. Fix any U, Pxyu £ "Pn(% x 3^xW) such that X — U — Y, 
Rx > 0, R Y > , S > 0, and u e T Pu . Define M x , My such that 



2 n(Rx-5) < M < 2 "flx 



2 n(R Y -5) < M < 2 nR Y 



Let X Mx = (x 1 ,X 2 ,...,X Mx ) and Y My = {y u Y 2 , ...,Y My ) be independent random variables, where 
XiS are uniformly distributed on Tp x]u (u), and YjS are uniformly distributed on Tp Y]u (u). 
Upper bound: By taking expectation over | [63l l, applying Lemma[4j and using the continuity of information 
measures, we get the desired upper bound. 

Lower bound: By taking expectation over (|65l l, applying Lemma 2J we get 



Ee(C, W) > Yl 2- n ( E x+* s ) 



y UXYXZ 



E 

V UXYXXZ : 



2—n{I v (XAXYXZ\U)-R x -75) 



V UXYY Z 



1 - 



2 -n(^ y +4a) 



1 - 



y U XYYY Z' 



E 



2~n(Iv(XY AXYXY Z\U)-R x -Ry -75) 



(212) 



Toward further simplification of this expression, we use the following lemma. 
Lemma 9. 

min /ypf AXYXZ\U) = I V (X AXYZ\U) 



(213) 



v U XY X XZ ■ 



Proof: Note that, for any V UXYXXZ , 

I V (X A XYXZ\U) = I V {X A XYZ\U) +I V (X A X\UXYZ), 

therefore, 

min I V (X A XYXZ\U) > I V (X A XYZ\U) = I V {X A XYZ\U). 

^UXYXXZ- 
^UXYX Z~^U XY X Z 

Now, consider V* v #& v „ defined as 

V uxyxxz( u > x ' y ' 5 ' z ) = V*|i/xrz( 5 l u > ^ y> z ) v x\uxyz(^ l u ' x -> f> z )%i-z(i, a;, y, z). 



Note that 7* - =7* 

UXXYZ UXXYZ 



, and X - (U, X, Y, Z) - X. Therefore, 

I V ,{X AXYXZ\U) = I V (X AXYZ\U) = I V (X A XYZ\U). 



(214) 



(215) 



(216) 



(217) 
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By combining ( 12151 1 and (|217b , the proof is complete. ■ 
Using the above lemma, the average probability of error can be bounded from below as 

^UXYXZ^^X ,n ^UXYYZ^Y.n *U XY XY Z ^ 'xY y n 

I(XAXYZ\U)>Rx+12S I(YaXYZ\U)>Ry + 12S I v (XYaXY\U)+I v (X AY\U)> 

Rx+Ry+146 

Using the continuity argument, the lower bound on the average error probability follows. ■ 
Proof: (Theorem [3) As was done in Theorem [T] for the point-to-point case, here, we will obtain higher 
error exponents for almost all codes by removing certain types from the constraint sets V x , V Y and V XY . 
Let us define the sets of n-types V x , V x and V XY as follows: 

Vuxyx '■ Vxu — V X u — Pxu, Vyu = Pyu 
y's ,, : ' : { Fu{Vuxy),Fu{V uxy ) <R x +Ry } (218) 

F x(V UXYX ) < R x + Ry 

Vuxyy '■ Vxu = Pxu, Vyu — V YU — Pyu 
V Y ,n - { FuiVuxY^FuiV^y) <Rx+Ry \ (219) 

Fy(V uxyy ) < Rx + Ry 

Vjixyxy '■ V UXYX ,V UXYX G V x , V UXYY ,V UXYY G vi 



yt A J V UXYXY ■ y UXYX> y UXYX c V X, y UXYY' V UXYY c V Y I (220) 

i f xy(V uxyxy ),F xy (V uxyxy ) < R x + R Y J 

Lemma 10. Let C — C x x C Y be one of the multiuser codes whose existence is asserted in the Typical random 
coding packing lemma. The following hold: 

VVuxyx e (V x , n ) c =► N X (C,V UXYX ) = 0, (221) 
V Vuxyy € (V Y , n ) c =► N Y (C, V uxyy ) = 0, (222) 
V Vuxyxy e (V XY , n ) c N XY (C, V UXYXY ) = 0. (223) 
Proof: Consider V UXYX G (V x n ) c . If V xu ^ Pxc/ or 7^ Pxc; or V y;7 ^ P yi7 , it is clear that 

Nx(C,V UXYX )=0. (224) 
Now, let us assume Fu(Vuxy) > Rx + Ry + 36. In this case, by using H78aL we conclude that 

M x M Y M x My 

Nu(C,V UXY ) < 2 -»(«*+*0 ^^^liwCu^y,-) < 1 ^EDv,^^^) = °> ( 225 ) 

i— 1 J — 1 i— 1 j — 1 

and as a result, Nu(C, Vuxy) = 0. Now, note that 

^ m x m y 

N X (C,V UXYX ) = ] ^EEE\xv,( u ' Xi ' y i>^ 

i=l j= 1 k^i 
Mx My 

^ E E E (u, x 4 , y,) 

2—1 j — 1 ky^i 

= 2 nR xN u (C,V UXY ) = 0, (226) 
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therefore, N X (C, V VXYX ) = 0. Similarly, if Fu{V UXY ) > R x + Ry + 35, 

M x M Y Mx My 

nu{c,v uxy ) < 2-"<**+ j *) ^j2j^i TVuXY (u, Xi , yj ) < i^EEH.^^-y^ ' (227) 

i—l j — 1 i—1 j — 1 

and as a result, Nu(Cx,Cy, V UX y) = 0- Also, note that 

Mx My 

#*(C7, Wx) = E E E ^ xyJf ( u > Vi> x *) 

Mx M Y 

* E E E i*w ^. y,o = 0^ (228) 

z—l j — 1 fc^i 

therefore, Nx(C, V UXYX ) = 0. If -FxC^t/jcyjc) > -Rx + Ry + 5(5, by the property of the code derived in 
Lemma |5j we observe that N x (Cx,Cy,V uxyx ) = 0. Similarly, by doing a similar argument, it can be 
concluded that 

If Vuxyy G 04,J C => ^(C, Wf) = 0, (229) 

and 

If Vuxyxy G (Vjc y , n ) c =► y^^) - 0. (230) 

■ 

Upper bound: We will follow the techniques used in Theorem [2] to provide lower and upper bounds 
on the average probability of error of almost all codes in the random coding ensemble. For this, we will 
use the results of Lemma [6] Consider any typical two-user code C = Cx x Cy whose existence was 
established in Lemma [5] Applying d63l on C, and using the continuity argument, we conclude that 

e(C W) < ^2 2 _ ™ [D(yz i xyul|w|yxyu)+/v(XAy|t/)+|/v ' ( * AXYZ|c/) ^- Rx| + " 5 ' 51 

:v x , n nv x , n 

2-n[D(V z ixYu\\W\V X Yu)+Iv(XA,Y\U) + \Iv(Y/\XYZ\U)-R Y \ + -58] 



2-n[D(V zlX Yu\\W\VxYu)+Iv(XAY\U) + \Iv(XAY\U)+Iv(XY^ 



XYZ\U)~Rx-R Y \ + -55] 



<- 2-"[- E T(-Rx,-Ri-,W,Pi7xi')-6(5] (231) 

whenever n > ni(|-Z|, \X\, \y\, \U\, 5), where E T (Rx, Ry,W, Pxyu) is defined in the statement of the 
theorem. 

Lower bound: In the following, we obtain a lower bound on the average error probability of code 
C = Cx x Cy- Applying ll65l l on C, then using (a) Lemma [5] and (b) the fact that for V £ V Xn , we have 
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Afj > 0, and similar such facts about A Y and A XY , we get 



v UXYXZ 



E 



2~n(Iv {X AXY X Z\U)-R X -7S) 



v UXYXXZ ■ 
XY X Z~^U XY X Z 



VuXYYZ 

evi„nvi, 



2-n(I v (YAXYYZ\U)-B.Y-7S) 



E 



v UXYXY Z 



XYYZ~ *U XYY Z 



E 



2-n(I v (XY AXYXY Z\U)-R x -Ry-75) 



(232) 



This expression can be simplified as follows. 



e(C, W) > 



I(XAXYZ\U)>R X + 12S 



2~ nE x + 



v UX YYz^ Ytn nv Ytn 

I(YaXYZ\U)>Ry + 125 



E 



7 v (XFAJf'r"|C/)+/v(XA'r"|C/)> 
Rx+Ry + US 



Using the continuity argument, the lower bound on the average error probability follows. ■ 
Proof: (Theorem D Fix W, Vxyu e V„(X X y X U) with X - U - Y, R x > 0, R Y > 0, 8 > 0, and 
u G Tpp . Let C* = C* x x Cy be the multiuser code whose existence is asserted in Lemma [6] Taking into 
account the given u, the a-decoding yields the decoding sets 

Dij = {z : a(u,x l ,y 3 ,z) < a(u,x fc ,y;,z) for all (k,l) ^ 

Let us define the collection of n-types V x n > ^Yn ^ Vf y „ as follows: 



V 



X,: 



v Y,n — 



V, 



UXYX 



UXYY ■ 



V, 



UXYXY 



V, 



Vxu = V xu = Pxu,Vyu — Pyu 
Fu(Vuxy),F u (V U x Y ) < min{R x ,R Y } 
F x(V UXYX ) < mm{R x ,R Y } 

Vxu = Pxu, Vyu = V Y u = 
Fu{V U xy),F u (V uxy ) < mm{Rx,R Y } 
Fy(V uxyy ) < mm{R x ,R Y } 

UXYX^UXYX G Vx - ) 'UXYY i^UXYY G ^" 



{ Fxy(V uxyxy ),F X y(V uxyxy ) < min{R x , Ry} 

Lemma 11. For the multiuser code C* — C* x x C Y , the following holds: 

vvuxyx e (n, n y => ^(c*, w*) = o, 

VVuxyxy G (Vl y ,„) c =► A^C*, Vxy) = 0. 



(233) 



(234) 



(235) 



(236) 
(237) 
(238) 



Proof: The proof is very similar to the proof of lemma [TTJl ■ 
The average error probability of C* can be obtained as follows in a similar way that used in the proof 
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of Theorem [2] and Theorem [3] 

e(C*,W)< ^2 2- nlD{Vz * XYuHwlVxYu)+Iv( - X/ ^ Y \ U) ~ 3S] 

V UX YXZ^X,n^X,n 

+ ^2 2~ n[D ( Vz t XYU ^ wlVxYu)+Iv( - XAY \ uy 

+ ^2 2~ n[D(Vz * XYullw \ VxYu)+Iv ( XA,Ylu) ~ 3S] . 



Vu 

XY,n' ' v XY,n 



y U XY XY Z 



Now using the continuity argument the statement of the theorem follows. 
Proof: (Theorem [5) For any V UXYXZ e V x , 

H V (XY\ZU) > H V (XY\ZU), 

therefore, by subtracting Hy(Y\ZU) form both sides of (|240l l, we can conclude that 

H V (X\U) - I V (X A YZ\U) > H V (X\U) - I V (X A YZ\U), 
Since Vxu = V xu = Pxu, the last inequality is equivalent to 

I V {X A YZ\U) < I V (X A YZ\U). 
Since I V (X A XYZ\U) > I V (X A YZ\U), it can be seen that for any V UXYXZ e V r x 

I V (X A XYZ\U) > I V (X A YZ\U). 

Moreover, since 

V x C {V UXYXZ : Vuxyz e V(Puxy)} 

it can be easily concluded that 

E r x (R x ,R Y ,W, Pxyu) > E% u (R x ,R Yl W, P XYU ). 
Similarly, for any V UXYYZ e Vy> 

H V (XY\ZU) > H V {XY\ZU). 
By using the fact that, Vyu — V YU = Pyu, it can be concluded that 

I V (Y AXYZ\U) > I V (Y AXZ\U). 

Since 

V Y C {V UXYYZ : Vuxyz e V(iW)} , 

we conclude that 

£^(ii x ,iV,W,P OT ) > E% U (R X ,R Y ,W,P XYU ). 
Similarly, we can conclude that, for any V UXYXYZ € V XY , 

I V {XY A XYZ\U) + I(X A Y\U) > I V {XY A Z\U) + I(X A Y\U). 
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Since 



V r XY C (Vfe : Vc/xrz £ V(iW)} , (250) 

it can be concluded that 

E XY (R x ,R Y ,W,P XYU )> E%\ (R x ,Ry,W,Pxyu)- (251) 

By combining {(A), d248l l and ||251| |, we conclude that (|96at holds. Similarly, we can prove that (|96b| 
and d96ct hold. ■ 
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